This Week in AI: Stability AI Unveils StableLM - Open-Source Language Models for All

Hey there, AI enthusiasts!

The latest news on groundbreaking advancements in artificial intelligence. Today, we're exploring Stability AI's latest release: the open-source language model StableLM.

Stability AI introduced StableLM, featuring 3 billion and 7 billion parameter Alpha models, with 15 billion to 65 billion parameter models coming soon. Developers can now freely access, use, and modify StableLM base models for commercial or research purposes under the CC BY-SA-4.0 license.

In 2022, Stability AI launched Stable Diffusion, a transparent, open, and scalable image model. With StableLM, they continue to democratize foundational AI technology. Capable of generating text and code, these efficient, high-performing models will power various downstream applications.

StableLM follows the open-sourcing of earlier language models, in collaboration with EleutherAI , a nonprofit research organisation. These include GPT-J, GPT-NeoX, and the Pythia suite, which trained on The Pile open-source dataset. The latest open-source models, like Cerebras-GPT and Dolly-2, build on these efforts.

The StableLM models are trained on an experimental dataset that's three times larger than The Pile, boasting a massive 1.5 trillion tokens of content. Despite their smaller size compared to GPT-3's 175 billion parameters, StableLM excels in conversational and coding tasks.

Check out some example below, produced by our 7 billion parameter fine-tuned model:

Stability AI is also releasing research models fine-tuned on StableLM with human feedback (RLHF), using a combination of five recent open-source datasets for conversational agents: Alpaca, GPT4All, Dolly, ShareGPT, and HH. These models are for research use only, under a noncommercial CC BY-NC-SA 4.0 license.

With StableLM, Stability AI demonstrates its commitment to transparent, accessible, and supportive AI technology. Open-sourcing their models fosters trust and encourages researchers to explore and refine AI performance. By designing models to run on local devices, Stability AI aims to make AI technology widely available and economically beneficial for all.

The StableLM models are now accessible via their GitHub repository, with a full technical report coming soon. Stay tuned for more news as Stability AI kicks off their crowd-sourced RLHF program and collaborates with community efforts like Open Assistant to develop an open-source dataset for AI assistants.

Here's to a future of open and accessible AI technology for everyone! ??

Until next time,

da sachin sharma