Organize | Kexin Zhu
Listing | CSDN (ID: CSDNnews)
As we all know, the AI image generation tool Stable Diffusion released by the start-up company Stability AI last year has become a revolutionary image model and has also enabled the rapid development of the AI "Vincent diagram".
Full of everyone's expectations for its "no longer limited to developing image generation" and "open source", on April 20, Stability AI announced the launch of an open source large language model (LLM) - StableLM.
(Source: screenshot of Stability AI official blog)
Small parameters, big training, open source is very popular
According to the official report of Stability AI, the StableLM model is still in the Alpha stage, with relatively few parameters, only 3 billion and 7 billion parameters, and models with 15 billion to 65 billion parameters will be launched later. As a ChatGPT-like model, the StableLM model can generate text and code and power a range of downstream applications, providing high performance for small but efficient models with proper training.
However, compared with models such as ChatGPT and Bard, the biggest "advantage" of StableLM is that it can be downloaded and deployed locally by everyone, even notebooks can run.
As Stability AI put it frankly in the report: "Language models will become the backbone of our digital economy, and we want everyone to have a say in their design."
StableLM is currently open-sourced on GitHub. Its high performance and low consumption quickly attracted a lot of attention. In less than a day, the star has reached nearly 6000 Stars.
Write copywriting, code, and support commercialization
It is worth mentioning that the release of StableLM is mainly due to Stability AI's experience in open-sourcing early language models at EleutherAI (a non-profit research center). Its language models include GPT-J, GPT-NeoX and Pythia suites, which are based on The Pile open source dataset training.
StableLM is trained on a new experimental dataset built on The Pile but tripled in size, containing 1.5 trillion labeled items. Despite having only 3-7 billion parameters (compared to GPT-3's 175 billion parameters), the richness of this dataset enables StableLM to achieve surprisingly high performance on dialogue and encoding tasks.
Stability AI also said that as a proof of concept, they also fine-tuned the model using the Stanford University Alpaca program, using a combination of five recent dialogue agent datasets: Stanford University's Alpaca, Nomic-AI's gpt4all, RyokoAI's ShareGPT52K dataset, Dolly from Databricks Labs and HH from Anthropic, and publish these models as StableLM-Tuned-Alpha.
Currently, on the official blog, Stability AI also shows examples of the 7 billion parameter fine-tuning model in the aspects of "text dialogue, content creation, and coding":
In terms of conversation, when asked "What would you say to your friends who are about to graduate from high school," StableLM replied, "You should be proud of your accomplishments, your jobs, your friendships, etc., and have hopes for your future."
Facing the task of "writing an email", StableLM can also clearly list the relevant content according to the format of the letter.
(Source: Stability AI official blog screenshot)
Even, StableLM can also complete the lyrics of a rap:
(Source: Stability AI official blog screenshot)
In the face of the problem of "using C language to write the meaning of life that can be computerized", StableLM joked in a humorous tone after "the answer failed": "The actual meaning of life will have more complex components, such as society, philosophy, etc. .”
(Source: Stability AI official blog screenshot)
Not only that, the StableLM model is also allowed to be commercialized, but developers need to comply with the CCBY-SA-4.0 license terms to freely inspect, use and adjust the StableLM base model.
"Generous open source" has been criticized
With the open source of StableLM on GitHub, netizens at home and abroad have also launched heated discussions, but what was never expected is that many netizens began to criticize the effect of StableLM after trying it out:
"I tried to use everything it generated, but in the end I didn't take it.";
"StableLM's 'open source' is the only advantage.";
"Its answer is really mediocre, not what I want."
However, some netizens helped explain:
"After all, the parameters of the Alpha version are currently small";
"I still believe that StableLM will improve in the near future";
"It has made me gratified to be generous and open source."
In addition, issues such as the security of the "open source" of the StableLM model have also attracted everyone's attention.
As we all know, after the release of Stable Diffusion last year, Stability AI was complained and sued by many artists. Therefore, many people expressed doubts about StableLM: "I don't know if there will be lawsuits", "I am afraid that plagiarism will happen again"...
At the same time, many people worry that someone will use open source to write copywriting for phishing links and coordinate software attacks.
In this regard, the Stability AI official stated: "We open source our models to increase transparency and promote trust. Researchers can 'drill down' to verify performance, study explainability techniques, identify potential risks and help develop safeguards."
Reference link:
https://stability.ai/blog/stability-ai-launches-the-first-of-its-stablelm-suite-of-language-models
https://techcrunch.com/2023/04/19/stability-ai-releases-chatgpt-like-language-models/?guccounter=1
https://news.ycombinator.com/item?id=35629127
https://github.com/Stability-AI/StableLM
