Summary:
- Both models, FreeWilly1 and FreeWilly2, are good at reasoning and understanding subtleties of language, and are validated by various benchmarks.
- The data generation process is inspired by the Microsoft approach, using high-quality instructions from specific datasets.
- The FreeWilly model shows superior performance despite being trained on a smaller dataset compared to previous models.
- FreeWilly2 beats GPT-4 in some domains and GPT-3 in most domains.
With Llama 2 and now FreeWilly, the open source community is on a winning streak. As the competition intensifies, there is no doubt that we will achieve faster developments in the field of artificial intelligence.
There is a very important branch in the field of artificial intelligence, which is natural language processing (NLP). Simply put, it is to enable computers to understand and generate natural languages used by humans, such as Chinese and English. The most advanced natural language processing technology at present is the large-scale language model (LLM). This technology uses massive text data to train an AI model that can capture language rules and knowledge. This model can be applied in various scenarios, such as chatting, searching, writing, translating, etc.
Currently the most well-known large-scale language model is the GPT series developed by OpenAI. The latest version of this series is called GPT-4, and its parameters may reach trillions or even trillions. Since it has not been officially announced, it can be inferred that GPT-4 is even larger from the 175 billion parameters of GPT-3.5. Parameters are variables used to store and adjust information in an AI model, and the more parameters, the more powerful the model. ChatGPT is an intelligent chat robot based on GPT-3.5 (175 billion parameters), which can have smooth, interesting and in-depth conversations with humans.
However, OpenAI's GPT family is not the only option. Recently, the LLM open source community has been very lively, and various models that are comparable to GPT have been continuously launched. A few days ago, a company called Stability AI and its subsidiary CarperAI jointly released two new large-scale language models, respectively. Called FreeWilly1 and FreeWilly2 . These two models are fine-tuned based on Meta's LLaMA series of models, capable of handling complex natural language understanding and reasoning tasks, and are open source and free, but the current authorization is only for research and cannot be used for business.
Meta is a company focusing on Meta-Learning and Meta-Data technologies. Meta-learning is a technique that enables AI models to quickly adapt to new tasks and environments. Metadata is data that describes the characteristics and relationships of the data itself. At the end of 2022, Meta released a series of large-scale language models that are commercially available, open source and free, high performance and low cost, and cross-platform adaptation, called the LLaMA series. The latest version of LLaMA 2 has 70 billion parameters, equivalent to about half of GPT-3.5.
FreeWilly1 is fine-tuned on the basis of LLaMA 65B (65 billion parameters), while FreeWilly2 is fine-tuned on LLaMA 2 70B (70 billion parameters). They all use a very advanced and efficient data generation and training method called the Orca Method . This method is a method proposed by Microsoft in 2022 to use a large-scale language model to generate training data itself and improve reasoning ability by gradually learning complex explanation processes.
According to this method, Stability AI generated 600,000 training data using language models of different quality and complexity, and used these data to fine-tune the LLaMA series of models. Although this data represents only 10% of the amount of data used by the original Orca method, it has achieved very significant results.
Finally, FreeWilly1 and FreeWilly2 show amazing performance on multiple public evaluation metrics, even surpassing ChatGPT.
- On HellaSwag, a natural language reasoning task that requires common sense and reasoning ability, FreeWilly2 achieved an accuracy rate of 86.4%, while ChatGPT was only 85.5%.
- In MMLU, a comprehensive evaluation that requires multi-tasking capabilities, FreeWilly2 achieved an accuracy rate of 68.8%, while ChatGPT was only 70.0%.
- On AGIEval, a comprehensive test for evaluating artificial general intelligence (AGI), FreeWilly2 achieved comparable or higher performance than ChatGPT on all subtasks except SAT math.
These achievements undoubtedly prove that FreeWilly1 and FreeWilly2 are powerful and advanced, and also provide a new possibility and opportunity for the open source community. Stability AI stated that they hope that these two models will bring infinite possibilities to the AI community and inspire new AI applications. At the same time, they also emphasized the importance of AI security, saying that they have conducted internal security tests on these two models and welcome external feedback and assistance.
If you are interested in FreeWilly1 and FreeWilly2, want to know more details or try to use them, you can visit the official website of Stability AI or the [Model Library] of Hugging Face. I believe you will be shocked and surprised by these two super powerful open source language models. Thank you for reading!
resource
GPT-4 Technical Report
https://arxiv.org/abs/2303.08774v3
Orca: Incremental Learning from Complex Interpretation Traces of GPT-4
https://arxiv.org/abs/2306.02707
If you are interested in this article, and you want to learn more about practical skills in the field of AI , you can pay attention to the "Technology Craze AI" public account . Here, you can see the latest and hottest dry goods articles and case tutorials in the AIGC field.