In-depth cooperation between Meta and Microsoft officially launched the next-generation open source large language model Llama 2 , and announced that it is free for research and commercial use.
Llama 2 paper address: Llama 2: Open Foundation and Fine-Tuned Chat Models
According to reports, compared with Llama 1, Llama 2 has 40% more training data, the context length is twice that of Llama 1 , and a group query attention mechanism is adopted. Specifically, the Llama 2 pre-trained model is trained on 2 trillion tokens, and the fine-tuned Chat model is trained on 1 million human-labeled data.
Meta says Llama 2 outperforms other models on a number of external benchmarks, including inference, coding, proficiency and knowledge tests.
Llama 2 includes Llama 2 and Llama 2-Chat, including 3 versions of 7 billion, 13 billion and 70 billion parameters, of which Llama 2-Chat is fine-tuned for two-way dialogue, and similar to ChatGPT, Llama 2-Chat has also experienced There are three stages: pre-training stage (PT), instruction fine-tuning (SFT), and reinforcement learning with human feedback (RLHF).
Meta said that Llama 2 is free for research and commercial use, and developers can download the model directly from the Llama 2 official website.
Address: https://ai.meta.com/resources/models-and-libraries/llama-downloads/
BTW, according to the licensing policy of Llama 2, if the number of monthly active users of the enterprise exceeds 700 million, it must apply for a license from Meta . Meta places strict limits on such authorizations.
In order to prevent the emergence of bad information and products such as deepfakes and pornographic chatbots brought about by the previous LLaMA leak, Llama 2 has been tested by the red team to avoid bad and harmful content from the model, and has developed guidelines and guidelines for developers. code.
Meta's move to release the first open source and commercially available large language model is undoubtedly a tough move against the two giants of OpenAI and Google. After all, OpenAI's GPT-4 and Google's PaLM 2 are both "technical confidentiality routes." Yann LeCun, Meta's chief scientist and winner of the Turing Award , believes that Meta's move may change the competitive landscape of the large-scale model industry
Microsoft embraces Meta, OpenAI,