Author | huggingface compiled | VK source | Github
This section describes how to save and reload the fine-tuning model (BERT, GPT, GPT-2 and Transformer-XL). You need to save three file types to reload the model has been fine:
- PyTorch model itself should be serialized saved model ( https://pytorch.org/docs/stable/notes/serialization.html#best-practices )
- Model configuration file is saved as a JSON file
- Glossary (BPE-based model and GPT and GPT-2 combined).
The default file names of these files are as follows:
- Model weights file:
pytorch_model.bin - Profiles:
config.json - Glossary file:
vocab.txton behalf of BERT and Transformer-XL,vocab.jsonon behalf of GPT / GPT-2 (BPE vocabulary), - On behalf of GPT / GPT-2 (BPE vocabulary) additional merge file:
merges.txt.
If you use the default file name to save the model, you can use from_pretrained () method to reload the model and tokenizer.
This is to save the model, configuration, and recommended method of configuration files. Words to the output_dirdirectory, and then re-loading model and tokenizer:
from transformers import WEIGHTS_NAME, CONFIG_NAME
output_dir = "./models/"
# 步骤1：保存一个经过微调的模型、配置和词汇表
#如果我们有一个分布式模型，只保存封装的模型
#它包装在PyTorch DistributedDataParallel或DataParallel中
model_to_save = model.module if hasattr(model, 'module') else model
#如果使用预定义的名称保存，则可以使用`from_pretrained`加载
output_model_file = os.path.join(output_dir, WEIGHTS_NAME)
output_config_file = os.path.join(output_dir, CONFIG_NAME)
torch.save(model_to_save.state_dict(), output_model_file)
model_to_save.config.to_json_file(output_config_file)
tokenizer.save_vocabulary(output_dir)
# 步骤2: 重新加载保存的模型
#Bert模型示例
model = BertForQuestionAnswering.from_pretrained(output_dir)
tokenizer = BertTokenizer.from_pretrained(output_dir, do_lower_case=args.do_lower_case) # Add specific options if needed
#GPT模型示例
model = OpenAIGPTDoubleHeadsModel.from_pretrained(output_dir)
tokenizer = OpenAIGPTTokenizer.from_pretrained(output_dir)If you want to use a specific path for each type of file, you can use another method to save and reload the model:
output_model_file = "./models/my_own_model_file.bin"
output_config_file = "./models/my_own_config_file.bin"
output_vocab_file = "./models/my_own_vocab_file.bin"
# 步骤1：保存一个经过微调的模型、配置和词汇表
#如果我们有一个分布式模型，只保存封装的模型
#它包装在PyTorch DistributedDataParallel或DataParallel中
model_to_save = model.module if hasattr(model, 'module') else model
torch.save(model_to_save.state_dict(), output_model_file)
model_to_save.config.to_json_file(output_config_file)
tokenizer.save_vocabulary(output_vocab_file)
# 步骤2: 重新加载保存的模型
# 我们没有使用预定义权重名称、配置名称进行保存，无法使用`from_pretrained`进行加载。
# 下面是在这种情况下的操作方法：
#Bert模型示例
config = BertConfig.from_json_file(output_config_file)
model = BertForQuestionAnswering(config)
state_dict = torch.load(output_model_file)
model.load_state_dict(state_dict)
tokenizer = BertTokenizer(output_vocab_file, do_lower_case=args.do_lower_case)
#GPT模型示例
config = OpenAIGPTConfig.from_json_file(output_config_file)
model = OpenAIGPTDoubleHeadsModel(config)
state_dict = torch.load(output_model_file)
model.load_state_dict(state_dict)
tokenizer = OpenAIGPTTokenizer(output_vocab_file)Original link: https://huggingface.co/transformers/serialization.html#serialization-best-practices
AI welcomes the attention Pan Chong station blog: http://panchuang.net/
OpenCV Chinese official document: http://woshicver.com/
Welcome attention Pan Chong blog resources Summary station: http://docs.panchuang.net/
