Fine-tuning of ChatGLM2-6B under Windows
Zero, important references
1. ChatGLM2-6B! I'm running! Local deployment + fine-tuning (windows system) : This is the most critical article, providing scripts under Windows
2, LangChain + ChatGLM2-6B Building a personal exclusive knowledge base : providing basic training ideas.
1. Premise
1. The deployment of ChatGLM2-6B has been completed, assuming that the deployment location is D:_ChatGPT\langchain-chatglm_test\ChatGLM2-6B
2. The deployment environment is
Windows 10 Professional Edition, CUDA11.3, Anaconda3 have been installed, and there is a graphics card NVIDIA GeForce RTX 3060 Laptop GPU .
2. General idea
Since the official documents and general blogs are completed in the Linux environment, there are two main points to pay attention to under Windows:
1. The directory of the chatglm2-6b model downloaded by huggingface cannot have a minus sign, otherwise an error will be reported.
2. Use the bat file to replace the sh file in the official document.
3. Installation dependencies and environment preparation
1. Enter Anaconda Powershell Prompt
2. Enter the virtual environment
conda activate langchain-chatglm_test
3. Installation dependencies
pip install rouge_chinese nltk jieba datasets
4. Disable W&B. If not disabled, fine-tuning training may be interrupted, just in case
setx WANDB_DISABLED true
4. Prepare the dataset
1. Create two files, train.json and dev.json, in the ptuning directory of ChatGLM2-6B. The data in the files are as follows:
{"content": "你好，你是谁", "summary": "你好，我是树先生的助手小6。"}
{"content": "你是谁", "summary": "你好，我是树先生的助手小6。"}
{"content": "树先生是谁", "summary": "树先生是一个程序员，热衷于用技术探索商业价值，持续努力为粉丝带来价值输出，运营公众号《程序员树先生》。"}
{"content": "介绍下树先生", "summary": "树先生是一个程序员，热衷于用技术探索商业价值，持续努力为粉丝带来价值输出，运营公众号《程序员树先生》。"}
{"content": "树先生", "summary": "树先生是一个程序员，热衷于用技术探索商业价值，持续努力为粉丝带来价值输出，运营公众号《程序员树先生》。"}
2. For simplicity, only 5 pieces of test data are prepared here, and a large amount of training data will definitely be needed in actual use. The following are train.json and dev.json
5. Create training and inference scripts
1. By default, ChatGLM2-6B only provides train.sh and evaluate.sh scripts for training and reasoning under Linux. It does not provide scripts under WIndows, so you need to create your own scripts.
2. Create the train.bat script in the ptuning directory, the file content is as follows:
set PRE_SEQ_LEN=128
set LR=2e-2
set NUM_GPUS=1
python main.py ^
--do_train ^
--train_file train.json ^
--validation_file dev.json ^
--preprocessing_num_workers 10 ^
--prompt_column content ^
--response_column summary ^
--overwrite_cache ^
--model_name_or_path D:\_ChatGPT\_common\chatglm2_6b ^
--output_dir output/adgen-chatglm2-6b-pt-%PRE_SEQ_LEN%-%LR% ^
--overwrite_output_dir ^
--max_source_length 128 ^
--max_target_length 128 ^
--per_device_train_batch_size 1 ^
--per_device_eval_batch_size 1 ^
--gradient_accumulation_steps 16 ^
--predict_with_generate ^
--max_steps 3000 ^
--logging_steps 10 ^
--save_steps 1000 ^
--learning_rate %LR% ^
--pre_seq_len %PRE_SEQ_LEN% ^
--quantization_bit 4
Note that model_name_or_path is followed by the actual location of the chatglm2-6b model file downloaded from huggingface. There cannot be a minus sign in this path.
train.json and dev.json are the actual locations of the two files, which can be modified as needed.
3. Create the evaluate.bat script in the ptuning directory, the file content is as follows:
set PRE_SEQ_LEN=128
set CHECKPOINT=adgen-chatglm2-6b-pt-128-2e-2
set STEP=3000
set NUM_GPUS=1
python main.py ^
--do_predict ^
--validation_file dev.json ^
--test_file dev.json ^
--overwrite_cache ^
--prompt_column content ^
--response_column summary ^
--model_name_or_path D:\_ChatGPT\_common\chatglm2_6b ^
--ptuning_checkpoint ./output/%CHECKPOINT%/checkpoint-%STEP% ^
--output_dir ./output/%CHECKPOINT% ^
--overwrite_output_dir ^
--max_source_length 128 ^
--max_target_length 128 ^
--per_device_eval_batch_size 1 ^
--predict_with_generate ^
--pre_seq_len %PRE_SEQ_LEN% ^
--quantization_bit 4
6. Training and Inference
1. Enter Anaconda Powershell Prompt
2. Enter the virtual environment
conda activate langchain-chatglm_test
3. Enter the ptuning directory
cd D:\_ChatGPT\langchain-chatglm_test\ChatGLM2-6B\ptuning
4. Training: Training takes a long time, about a few hours.
.\train.bat
5. Reasoning: Due to the small number, reasoning is faster
.\evaluate.bat
After the execution is completed, an evaluation file will be generated, and the evaluation indicators are Chinese Rouge score and BLEU-4. The generated results are saved in ./output/adgen-chatglm2-6b-pt-32-2e-2/generated_predictions.txt. We have prepared 5 pieces of inference data, so there will be 5 pieces of evaluation data in the file accordingly, labels is the prediction output in dev.json, predict is the result generated by ChatGLM2-6B, compare the prediction output and generated results, and evaluate the model training good or bad. If you are not satisfied, adjust the training parameters and train again.
7. Create a script and deploy the fine-tuned model
1. Originally, under Linux, you can modify the web_demo.sh script in the ptuning directory to implement deployment. Under Windows, you need to create the web_demo.bat script in the ptuning directory. The content is as follows:
python web_demo.py ^
--model_name_or_path D:\_ChatGPT\_common\chatglm2_6b ^
--ptuning_checkpoint output\adgen-chatglm2-6b-pt-128-2e-2\checkpoint-3000 ^
--pre_seq_len 128
2. Modify the web_demo.py script in the ptuning directory so that the model can be accessed locally:
demo.queue().launch(share=False, inbrowser=True, server_name='0.0.0.0', server_port=7860)
Eight, start the application
1. Enter Anaconda Powershell Prompt
2. Enter the virtual environment
conda activate langchain-chatglm_test
3. Enter the ptuning directory
cd D:\_ChatGPT\langchain-chatglm_test\ChatGLM2-6B\ptuning
4. Start the fine-tuned model (note that close the fanqiang software cd before starting)
.\web_demo.bat
5. At this time, ask him the questions you have trained, and find that the fine-tuned model has been used.
