Introduction to fine-tuning
Fine-tune is the English word for fine-tuning that is often said in LLM. Training a model in LLM is a time-consuming, laborious, and expensive task. Due to the large scale of the model and many parameters, it will take up a lot of time and computing resources during the training process, especially for a complex task. If you want to improve the capabilities of the model , it requires a large amount of data for training, which requires more investment. So if there is a ready-made model that is similar to the problem we want to think about, then we can achieve the capabilities we want by fine-tuning based on this model. Fine-tuning is to train again on a large pre-trained model using data from some specific fields, so that the pre-trained model can adapt to specific fields and specific tasks. During the fine-tuning process, the parameters of the model are adjusted based on new data to meet the needs of specific fields and tasks.
There are also some different methods of fine-tuning. Full fine-tuning means that all model parameters will be adjusted during the fine-tuning process. This also determines that comprehensive fine-tuning requires more support in terms of investment in computing resources and data scale requirements. Comprehensive fine-tuning can easily cause catastrophic forgetting because all parameters are adjusted. Catastrophic forgetting means that the model obtained after comprehensive fine-tuning performs very poorly on the original task, so much so that it reaches an unacceptable level. decline.
Then parameter-efficient fine-tuning (PEFT) is another fine-tuning method. During the fine-tuning process, PEFT will only adjust some model parameters or add some new parameters without modifying the original parameters, thereby reducing the requirements for computing investment and data scale. PEFT has a comprehensive balance in parameter efficiency, memory efficiency, training speed, model performance and inference cost. PEFT is divided into selective method (selective), reparametrization-based method (reparametrization-based), and additive method (additive). The selection method will select some of the initial parameters of the LLM for fine-tuning, because when selecting again, you can select some components and some parameters.