[Large Model Practice] ChatGLM3 fine-tuning dialogue model (5)

ChatGLM3 is a new generation of dialogue pre-training model jointly released by Zhipu AI and Tsinghua University KEG Laboratory. ChatGLM3-6B is an open source model in the ChatGLM3 series. On the basis of retaining many excellent features of the previous two generations of models such as smooth dialogue and low deployment threshold, ChatGLM3-6B introduces the following features:

More powerful basic model: ChatGLM3-6B-Base, the basic model of ChatGLM3-6B, uses more diverse training data, more sufficient training steps, and a more reasonable training strategy. Evaluations on data sets from different perspectives such as semantics, mathematics, reasoning, code, and knowledge show that ChatGLM3-6B-Base has the strongest performance among basic models below 10B.
More complete function support: ChatGLM3-6B adopts a newly designed Prompt format, in addition to normal multi-round conversations. At the same time, it natively supports complex scenarios such as tool calling (Function Call), code execution (Code Interpreter), and Agent tasks.
More comprehensive open source sequence: In addition to the dialogue model ChatGLM3-6B, the basic model ChatGLM3-6B-Base and the long text dialogue model ChatGLM3-6B-32k are also open sourced. All the above rights are completely open to academic research, and free commercial use is also allowed after filling in the questionnaire for registration.

This article introduces an example of fine-tuning ChatGLM3-6B on the multi-turn dialogue model ToolAlpaca data set. Please make sure I complete the environment installation and model download parts in [Large Model Practice] ChatGLM3 Installation and Experience (4) .

1. Prepare the data set

In order to train a multi-turn dialogue model, the data must first be prepared according to the data format of chatglm-6b. The dialogue and tool capability data format of the fine-tuning model of chatglm-6b is as follows:

[
   {
      "tools": [
         // available tools, format is not restricted
      ],
      "conversations": [
         {
            "role": "system",
            "content": "<system prompt text>"
         },
         {
            "role": "user",
            "content": "<user prompt text>"
         },
         {
            "role": "assistant",
            "content": "<assistant thought to text>"
         },
         {
            "role": "tool",
            "name": "<name of the tool to be called",
            "parameters": {
               "<parameter_name>": "<parameter_value>"
            },
            "observation": "<observation>"
            // don't have to be string
         },
         {
            "role": "assistant",
            "content": "<assistant response to observation>"
         },
         // ... Muti Turn
         {
            "role": "user",
            "content": "<user prompt text>"
         },
         {
            "role": "assistant",
            "content": "<assistant response text>"
         }
      ]
   }
   // ...
]

 Next, download the ToolAlpaca data set . There are 3 json files in the data directory, as shown in the following figure:

Enter the finetune_chatmodel_demo directory:

cd finetune_chatmodel_demo

Next, use the data processing tool of chatglm3-6b to convert the format of the data set into chatglm3 format (replace the path of the json file with your own training data path):

python scripts/format_tool_alpaca.py --path "data/toolalpaca/train_data.json"

After the processing is completed, the formatted_data directory will appear under the finetune_chatmodel_demo directory, which contains the processed data, as shown below:

2. P-Tuning v2 fine-tuning

This tutorial uses P-Tuning v2 for fine-tuning. Replace the BASE_MODEL_PATH of ChatGLM3/finetune_chatmodel_demo/scripts/finetune_pt_multiturn.sh with your own path to chatglm-6b):

#! /usr/bin/env bash

set -ex

PRE_SEQ_LEN=128
LR=2e-2
NUM_GPUS=1
MAX_SEQ_LEN=2048
DEV_BATCH_SIZE=1
GRAD_ACCUMULARION_STEPS=16
MAX_STEP=1000
SAVE_INTERVAL=500

DATESTR=`date +%Y%m%d-%H%M%S`
RUN_NAME=tool_alpaca_pt

BASE_MODEL_PATH=chatglm3-6b
DATASET_PATH=formatted_data/tool_alpaca.jsonl
OUTPUT_DIR=output/${RUN_NAME}-${DATESTR}-${PRE_SEQ_LEN}-${LR}

mkdir -p $OUTPUT_DIR

torchrun --standalone --nnodes=1 --nproc_per_node=$NUM_GPUS finetune.py \
    --train_format multi-turn \
    --train_file $DATASET_PATH \
    --max_seq_length $MAX_SEQ_LEN \
    --preprocessing_num_workers 1 \
    --model_name_or_path $BASE_MODEL_PATH \
    --output_dir $OUTPUT_DIR \
    --per_device_train_batch_size $DEV_BATCH_SIZE \
    --gradient_accumulation_steps $GRAD_ACCUMULARION_STEPS \
    --max_steps $MAX_STEP \
    --logging_steps 1 \
    --save_steps $SAVE_INTERVAL \
    --learning_rate $LR \
    --pre_seq_len $PRE_SEQ_LEN 2>&1 | tee ${OUTPUT_DIR}/train.log

Before fine-tuning the training, install another dependent library (otherwise the training will report an error):

pip install astunparse

Next, start fine-tuning training:

bash scripts/finetune_pt_multiturn.sh

As you can see, training is started:

Video memory consumption is close to 17GB:

End of training:

3. Deployment

After obtaining the fine-tuned model, return to the composite_demo directory:

cd ../composite_demo

It can be deployed very simply (MODEL_PATH is changed to the weight path of your own chatglm3-6b, PT_PATH points to the fine-tuning path, and the PT_PATH path is in the log after training):

MODEL_PATH="chatglm3-6b" PT_PATH="output/tool_alpaca_pt-20231227-061735-128-2e-2" streamlit run main.py 

The effect is as follows: