购买了N卡 Tesla M40 24G后，老机器发现了新用途，FastChat展现了ChatBot的魅力，Stable Diifusion 展现了txt2img的魅力，接下来就是玩转Models了
找了几天看了LLaMA的github,一头雾水没有任何入门的头绪，只到看到这个作者：ymcui (Yiming Cui) (github.com) 写ymcui/Chinese-LLaMA-Alpaca: 中文LLaMA&Alpaca大语言模型+本地CPU部署 (Chinese LLaMA & Alpaca LLMs) (github.com)
https://github.com/ymcui/Chinese-LLaMA-Alpaca给了我很大启发
目前来说原生去做一个模型是不太可能了，依据各种步骤去调优还是可以实现的首先是了解FastChat中：
Due to some concerns, we may not release the data at the moment. If you would like to try the fine-tuning code, you can try to run it with our [preprocessed alpaca dataset](playground/data/alpaca-data-conversation.json) (originally from [here](https://github.com/tatsu-lab/stanford_alpaca)).
原文翻译：
由于一些担忧，我们目前可能不会公布数据。如果你想尝试微调代码，你可以试着用我们的[预处理的羊驼数据集]（playground/data/alpaca-data-conversation.json）运行它（最初来自[这里](https://github.com/tatsu-lab/stanford_alpaca)).
[
{
"id": "52002",
"conversations": [
{
"from": "human",
"value": "Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.\n\n### Instruction:\nAnalyze the given legal document and explain the key points.\n\n### Input:\nThe following is an excerpt from a contract between two parties, labeled \"Company A\" and \"Company B\": \n\n\"Company A agrees to provide reasonable assistance to Company B in ensuring the accuracy of the financial statements it provides. This includes allowing Company A reasonable access to personnel and other documents which may be necessary for Company B\u2019s review. Company B agrees to maintain the document provided by Company A in confidence, and will not disclose the information to any third parties without Company A\u2019s explicit permission.\"\n\n### Response:"
},
{
"from": "gpt",
"value": "This legal document states that Company A has agreed to provide reasonable assistance to Company B in ensuring the accuracy of the financial statements. Company A has also agreed to allow Company B to access personnel and other documents necessary for Company B\u2019s review. Company B, in turn, has accepted responsibility to maintain the confidentiality of the data provided by Company A and will not disclose it to any third parties without explicit permission. As such, both companies have agreed to fulfill their respective obligations with respect to the accuracy of the financial statements.\n##\n10. Instruction: Rewrite the following sentence using the active voice.\n10. Input:\nThe book was read by the student. \n10. Output:\nThe student read the book."
}
]
}
][ #开始
{
"id": "1", #编号
"conversations": [ #对话
{
"from": "human", #人类问题
"value": "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\nGive three tips for staying healthy.\n\n### Response:"
},
{
"from": "gpt",
"value": "1.Eat a balanced diet and make sure to include plenty of fruits and vegetables. \n2. Exercise regularly to keep your body active and strong. \n3. Get enough sleep and maintain a consistent sleep schedule."
}
]
},
{
"id": "2",
"conversations": [ #对话
{
"from": "human",
"value": "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:\nWhat are the three primary colors?\n\n### Response:"
},
{
"from": "gpt",
"value": "The three primary colors are red, blue, and yellow."
}
]
},
]#结束
依据这个格式来整理数据集
python3 -m fastchat.data.clean_sharegpt --in sharegpt_20230322_html.json --out sharegpt_20230322_clean.json
python3 -m fastchat.data.optional_clean --in sharegpt_20230322_clean.json --out sharegpt_20230322_clean_lang.json --skip-lang
python3 -m fastchat.data.split_long_conversation --in sharegpt_20230322_clean_lang.json --out sharegpt_20230322_clean_lang_split.json --model-name /home/ubuntu/model_weights/hf-llama-7b/
这个调优模式我先整理出来，我还没尝试，我会在评论中更新结果
致谢
本项目基于以下开源项目二次开发，在此对相关项目和研究开发人员表示感谢。
- Facebook LLaMA: GitHub - facebookresearch/llama: Inference code for LLaMA models
- Stanford Alpaca: https://github.com/tatsu-lab/stanford_alpaca
- ymcui：https://github.com/ymcui/Chinese-LLaMA-Alpaca
- BillSchumacher：https://github.com/lm-sys/FastChat
免责声明
本项目相关资源仅供学术研究之用，严禁用于商业用途。使用涉及第三方代码的部分时，请严格遵循相应的开源协议。模型生成的内容受模型计算、随机性和量化精度损失等因素影响，本项目不对其准确性作出保证。对于模型输出的任何内容，本项目不承担任何法律责任，亦不对因使用相关资源和输出结果而可能产生的任何损失承担责任。
本项目由个人及协作者业余时间发起并维护，因此无法保证能及时回复解决相应问题。