LLM: ChatGLM-6B model for P-Tunning training records and parameter explanations

model training

First, let me explain the purpose of training: to provide local question-and-answer knowledge files, and after training, it can answer according to the semantics close to the original text, similar to a question-and-answer robot.

step

1. Dependencies needed to install fine-tuning:

pip install rouge_chinese nltk jieba datasets

1. Prepare the training dataset:

The data set needs to be in jsonline format. If it is a single-round dialogue, you need to set two input and output fields (the field name can be customized, and you can specify it through parameters in the training script). If it is a multi-round dialogue, you need to specify an additional history field.

Take a single-turn dialogue as an example:

{"question":"南京未来菁英训练营的报名年龄？","answer":"9-15岁，向下浮动2岁，向上浮动3岁。"}
{"question":"南京未来菁英训练营的接待标准是？","answer":"住宿：211高校、正餐餐标45元/人（5荤5素1汤1主食）。"}

1. Prepare the training script: tran.sh

PRE_SEQ_LEN=64
LR=2e-2

CUDA_VISIBLE_DEVICES=0 python main.py \
    --do_train \
    --train_file qa-all.json \
    --validation_file qa-dev.json \
    --prompt_column question \
    --response_column answer \