Use peft's lora to fine-tune MAE

Internet 2023-06-21 07:45:11 views: null

1. peft download

https://github.com/huggingface/peft

Two, peft installation

打开cmd
到达安装目录
python setup.py build
python setup.py install

3. Make finetune.py

Guess you like

Origin blog.csdn.net/gaoenyang760525/article/details/130114265

Use peft's lora to fine-tune MAE

TensorFlow learning: Use the official model for image classification and use your own data to fine-tune the model

How to use GPT3 to fine-tune your own model in Windows environment

Use Transformers' Trainer to fine-tune pre-trained large models in PyTorch

[AWS Series] Use Amazon SageMaker to fine-tune and deploy the ChatGLM model

BERT model fine-tuning in practice: Use Transformers to fine-tune the BERT model for question answering and text classification tasks

Large model-fine-tuning technology: unified framework (unified Adapter-Tuning, Prefix-Tuning, LoRA) [freeze large model parameters, fine-tune the newly inserted parameter layer]

Introduction to Fine-tune

[NLP] LLM efficient fine-tuning (PEFT)--LoRA

LLMs PEFT技术1：LoRA Parameter efficient fine-tuning PEFT techniques 1: LoRA Low rank Adaptation

[GPT] Can you refine GPT with a small budget? Let's train/fine-tune medium-sized GPT together

With 65 billion parameters, 8 GPUs can fine-tune the parameters of the large model. The latest paper of Qiu Xipeng's team is here!

65 billion parameters, 8 GPUs can fine-tune: Qiu Xipeng's team has lowered the threshold of large models

65 billion parameters, 8 GPUs can fine-tune all parameters: Qiu Xipeng's team has lowered the threshold of large models

LLMs: LLaMA Efficient Tuning (an efficient tool that can efficiently fine-tune [full parameters/LoRA/QLoRA] mainstream large models [ChatGLM2/LLaMA2/Baichuan, etc.] [pre-training + instruction supervision fine-tuning +

How to use the pre-trained model to fine-tune the model (such as freezing certain layers, setting different learning rates for different layers, etc.)

Peft library combat (3): Lora fine-tuning mt0/bloom (GPT generation)

Hand-written deep learning (21): Fine-tuning Stable Diffusion with PEFT LoRA

LoRA fine-tuning

Website optimization [SEO] how to fine-tune the site to enhance the rankings

Transfer learning, fine-tune the parameters and local recovery

Face detection and face recognition principle trimming (Fine-tune) principle

Pytorch fixed parameters-model pretrain and fine-tune

Exclusive | When to fine-tune a large language model?

How to fine-tune the Chinese-Vicuna-7b model

An attempt to fine-tune collection questions and answers using openai

Fine-tune and deploy the ChatGLM model using Amazon SageMaker

How to Fine-Tune a Donut Model - Usage Example

Baichuan2 source code analysis fine-tune/fine-tune.py (1)

LLM-Project Detailed Explanation-Chinese-LLaMA-AIpaca (1): LLM+LoRa fine-tuning acceleration technology principle and PEFT-based hands-on practice: some thoughts and a complete case of mt0-large+lora

Recommended

Ranking

error: (-215:Assertion failed) !_img.empty() in function ‘cv::imwrite‘

Database migration between Navicat servers

Minimum number of rotation of the array: Array

balenaEtcher for mac (make a boot disk software) v1.5.67

Custom processing serialization and deserialization in jackson

Mu-en-mask system development software

Mastering Regular Expressions

Find mileage Java--

Web pages can not directly concern the public micro-channel number how to do? A key to arouse public concern number of micro-channel solutions

[CodeForces - 739B] Alyona and a tree Tree + [difference] + bipartite

Daily

More

2024-05-12(28)

2024-05-11(32)

2024-05-10(34)

2024-05-09(32)

2024-05-08(18)

2024-05-07(34)

2024-05-06(6)

2024-05-05(0)

2024-05-04(18)

2024-05-03(8)