[DL] LoRA study notes

2 Source code learning

2.1 Study materials

[Video] "[LLMs Practice] 04 PEFT/LoRA Source Code Analysis"

• PEFT: A library of Parameter-Efficient Fine-Tuning (PEFT) developed by the huggingface team, which can adapt pre-trained language models (PLM) to various downstream tasks without fine-tuning all model parameters superior. The PEFT library supports a variety of efficient fine-tuning methods, such as Low-Rank Adaptation (LoRA), Prefix Tuning (Prefix Tuning), Adaptive Budget Allocation (AdaLoRA), etc.