The basic concept of SFT supervised fine-tuning
SFT (Supervised Fine-Tuning) supervised fine-tuning refers to pre-training a neural network model on the source data set, that is, the source model. Then create a new neural network model, the target model. The target model replicates all the model design and its parameters on the source model except the output layer. These model parameters contain the knowledge learned on the source dataset, and this knowledge is also applicable to the target dataset. The output layer of the source model is closely related to the labels of the source dataset, so it is not adopted in the target model. When fine-tuning, add an output layer whose output size is the number of categories of the target data set to the target model, and initialize the model parameters of this layer randomly. When training the target model on the target dataset, the output layer will be trained from scratch, and the parameters of the remaining layers will be fine-tuned based on the parameters of the source model.
1.2 Steps of supervised fine-tuning
Specifically, supervised fine-tuning includes the following steps:
Pre-training: First train a deep learning model on a large-scale data set, such as using self-supervised learning or unsupervised learning algorithms for pre-training;
Fine-tuning: The pre-trained model is fine-tuned using the training set of the target task. Usually, only a part of the layers in the pre-trained model are fine-tuned, such as only the last few layers of the model or some intermediate layers. In the fine-tuning process, the model is optimized through the back-propagation algorithm to make the model perform better on the target task;
Evaluation: Use the test set of the target task to evaluate the fine-tuned model to obtain the performance index of the model on the target task.
1.3 Features of supervised fine-tuning
Supervised fine-tuning can use the parameters and structure of the pre-trained model to avoid training the model from scratch, thereby accelerating the training process of the model and improving the performance of the model on the target task. Supervised fine-tuning has been widely used in computer vision, natural language processing and other fields. However, supervision also has some disadvantages. First, a large amount of labeled data is required for fine-tuning the target task. If the labeled data is insufficient, it may lead to poor performance of the fine-tuned model. Second, since the parameters and structure of the pre-trained model have a great impact on the performance of the fine-tuned model, it is also important to choose an appropriate pre-trained model.
Author: Ting, Artificial Intelligence https://www.bilibili.com/read/cv25060283 Source: bilibili