Llama has long been hailed as the most powerful open source large model in the AI community. However, due to the restrictions of the open source agreement, it has not been free for commercial use. However, that all changed on July 19th, when Meta finally released the long-awaited free commercial version Llama2. Llama2 is a pre-trained large language model developed by Meta AI, which can accept any natural language text as input and generate textual output. Llama2-xb-chat is an optimized model based on Llama2-xb in dialogue scenarios. It currently surpasses other open source dialogue models in most evaluation indicators, and is comparable to some popular closed source models (such as ChatGPT, PaLM).
official introduction
The Llama 2 model series released by Meta includes three parameter versions of 7 billion, 13 billion and 70 billion. In addition, they also trained a 34 billion parameter version, but it was not released, only mentioned in the technical report. According to the official introduction, compared with its predecessor Llama 1, Llama 2 has 40% more training data, doubled the context length, and adopted a group query attention mechanism. Specifically, the Llama 2 pre-training model is trained on 2 trillion tokens, while the fine-tuned Chat model is trained on 1 million human-labeled data.
Published evaluation results show that Llama 2 outperforms other open source language models on a number of external benchmarks including inference, coding, proficiency and knowledge tests.
model deployment
Meta provides download links for all models on Huggingface: https://huggingface.co/meta-llama
pre-trained model
The Llama2 pre-training model includes three versions: 7B, 13B and 70B
| model name | model load name | download link |
|---|---|---|
| Llama2-7B | meta-llama/Llama-2-7b-hf | Model download |
| Llama2-13B | meta-llama/Llama-2-13b-hf | Model download |
| Llama2-70B | meta-llama/Llama-2-70b-hf | Model download |
Chat model
The Llama2-Chat model is supervised and fine-tuned based on the pre-trained model, which has stronger dialogue ability
| model name | model load name | download link |
|---|---|---|
| Llama2-7B-Chat | meta-llama/Llama-2-7b-chat-hf | Model download |
| Llama2-13B-Chat | meta-llama/Llama-2-13b-chat-hf | Model download |
| Llama2-70B-Chat | meta-llama/Llama-2-70b-chat-hf | Model download |
Alibaba Cloud Machine Learning Platform PAI
The machine learning platform PAI (Platform of Artificial Intelligence) is aimed at enterprise customers and developers, providing lightweight and cost-effective cloud-native machine learning, covering PAI-DSW interactive modeling, PAI-Studio drag-and-drop visual modeling, PAI- The whole process from DLC distributed training to PAI-EAS model online deployment.
PAI platform deployment
Today, the PAI platform also supports Llama2-7b, providing related images that can be deployed directly. After the model is deployed, users can directly interact with model inference on the web page through the "View Web Application" button on the service details page. Let's experience it!
After the deployment is complete:
Go to the web page to test it out:
In addition, direct reasoning through the API form is also supported, but you need to go to the EAS service and update the service running command to python api/api_server.py --port=8000 --model-path=<the previously filled model-path>. The body of the service request is input in text/plain format or application/json format, and the returned data is in text/html format. The following is an example format for sending a request:
{"input_ids": "List the largest islands which begin with letter 's'.","temperature": 0.8,"max_length": 5120,"top_p": 0.9}
API details
The LLAMA2 model API call can only be used after the "application experience" is passed, otherwise the API call will return an error status code. The following example shows the code that calls the LLAMA2 model to respond to a user command.
Python
# For prerequisites running the following sample, visit https://help.aliyun.com/document_detail/611472.html
from http import HTTPStatus
from dashscope import Generation
def simple_sample():
# 模型可以为模型列表中任一模型
response = Generation.call(model='llama2-7b-chat-v2',
prompt='Hey, are you conscious? Can you talk to me?')
if response.status_code == HTTPStatus.OK:
print('Result is: %s' % response.output)
else:
print('Failed request_id: %s, status_code: %s, code: %s, message:%s' %
(response.request_id, response.status_code, response.code,
response.message))
if __name__ == '__main__':
simple_sample()
example response
{"text": "Hey, are you conscious? Can you talk to me?\n[/Inst: Hey, I'm not sure if I'm conscious or not. I can't really feel anything or think very clearly. Can you tell me"}
HTTP call interface
curl --location 'https://dashscope.aliyuncs.com/api/v1/services/aigc/text-generation/generation' \
--header 'Authorization: Bearer <your-dashscope-api-key>' \
--header 'Content-Type: application/json' \
--data '{
"model": "llama2-7b-v2",
"input":{
"prompt":"Hey, are you conscious? Can you talk to me?"
}
}'
example response
{
"output":{
"text":"Hey, are you conscious? Can you talk to me?\nLeaders need to be conscious of what’s going on around them, and not just what’s happening within their own heads.\nThis means listening to your team."
},
"request_id":"fbd7e41a-363c-938a-81be-8ae0f9fbdb3d"
}
As time goes by, applications based on the Llama2 open source model are expected to spring up like mushrooms in China. This trend reflects a shift from relying on external technologies to self-development, which not only meets our specific needs and goals, but also avoids the risks of relying on external technologies. Therefore, we are more looking forward to seeing the emergence of excellent, independent, and autonomous large models, which will promote the development and progress of our AI technology.
More in-depth content will be summarized after subsequent study