Read and store

Part of the study notes of "Practice Deep Learning pytorch" is only for your own review.

Read and store

In practice, we sometimes need to deploy the trained model to many 不 devices . In this case, we can
store the model parameters trained in the memory on the hard disk for subsequent reading. 

 Read and write TENSOR

You can directly use the save function and load function to store and read Tensor respectively . save uses Python 's pickle utility to serialize objects , and then save the sequenced objects to disk . Use  save to save various objects, including models, tables, and dictionaries. And laod using pickle unpickling ⼯ object files with the file pickle deserialization of memory.

The following example creates a Tensor variable量x and saves it in a file with the same file name as x.pt.

import torch
from torch import nn
x = torch.ones(3)
torch.save(x, 'x.pt')

# 然后我们将数据从存储的文件读回内存。
x2 = torch.load('x.pt')
x2

Output: tensor([1., 1., 1.]) 

You can also store a Tensor list and read it back into memory.

y = torch.zeros(4)
torch.save([x, y], 'xy.pt')
xy_list = torch.load('xy.pt')
xy_list

Output:

[tensor([1., 1., 1.]), tensor([0., 0., 0., 0.])] 

Store and read a dictionary mapping from character 串 to Tensor .

torch.save({'x': x, 'y': y}, 'xy_dict.pt')
xy = torch.load('xy_dict.pt')
xy

Output: {'x': tensor([1., 1., 1.]),'y': tensor([0., 0., 0., 0.])) 

Read-write model

state_dict

In PyTorch , the learnable parameters of the Module (ie weights and deviations), the module model is included in the parameters ( accessed via model.parameters() ). state_dict is a dictionary object implied from the parameter name to the parameter Tesnor.

class MLP(nn.Module):
    def __init__(self):
        # 初始化
        super(MLP, self).__init__()
        # 隐藏层
        self.hidden = nn.Linear(3, 2)
        # 激活函数
        self.act = nn.ReLU()
        # 输出层
        self.output = nn.Linear(2, 1)
    def forward(self, x):
        a = self.act(self.hidden(x))
        return self.output(a)
net = MLP()
# state_dict 是⼀个从参数名称隐射到参数 Tesnor 的字典对象。
net.state_dict()

Output:

OrderedDict([('hidden.weight', tensor([[ 0.2448, 0.1856, -0.5678],
[ 0.2030, -0.2073, -0.0104]])),
('hidden.bias', tensor([-0.3117, -0.4232])),
('output.weight', tensor([[-0.4556, 0.4084]])),
('output.bias', tensor([-0.3573]))])

Note that only layers with learnable parameters (convolutional layer, linear layer, etc.) have entries in state_dict . Optimization (optim) also has a state_dict which contains information about the state of the optimizer and the hyperparameters used.

optimizer = torch.optim.SGD(net.parameters(), lr=0.001,
momentum=0.9)
optimizer.state_dict()

输出：{'param_groups': [{'dampening': 0,
'lr': 0.001,
'momentum': 0.9,
'nesterov': False,
'params': [4736167728, 4736166648, 4736167368, 4736165352],
'weight_decay': 0}],
'state': {}}

Save and load models

There are two common methods for saving and loading training models in PyTorch:
1. Only save and load model parameters (state_dict );
2. Save and load the entire model.

1. Save and load state_dict (recommended method)

Save:

torch.save(model.state_dict(), PATH) # 推荐的⽂件后缀名是pt或pth 

load:

model = TheModelClass(*args, **kwargs)
model.load_state_dict(torch.load(PATH))

2. Save and load the entire model

Save:

torch.save(model, PATH) 

load:

model = torch.load(PATH) 

Let's experiment with the recommended method:

X = torch.randn(2, 3)
Y = net(X)
PATH = "./net.pt"
# 保存
torch.save(net.state_dict(), PATH)
net2 = MLP()
# 加载
net2.load_state_dict(torch.load(PATH))
Y2 = net2(X)
Y2 == Y

Output:

tensor([[1],
[1]], dtype=torch.uint8)

summary

• Tensor can be read and written easily through the save function and load function.
• Through the save function and load_state_dict function, the parameters of the model can be read and written easily.