pytorch prints model parameters, freezes training and other operations

Ready to work

import torch.optim as optim
import torch
import torchvision.models as models
device=torch.device("cuda" if torch.cuda.is_available() else "cpu")
model=models.resnet50(pretrained=False).to(device)
optimizer=optim.Adam(model.parameters(),0.01)
scheduler = optim.lr_scheduler.MultiStepLR(optimizer, milestones=[3, 6, 10, 15, 21], gamma=1/3.)

#--------------------------------

View the parameters of the model

method one

Through the torchsummary package, first install this package,

pip install torchsummary

then

from torchsummary import summary
summary(model,input_size=(3,416,416))

result:

Method Two

Use model.named_parameters()

sum=0
for name,param in model.named_parameters():
    num=1
    for size in param.shape:
        num *= size
    sum+=num
    print("{:30s} : {}".format(name,param.shape))    
print("total param num {}".format(sum))

result:

Method Three

Use model.parameters() directly, but the parameter names are invisible in this way.

for param in model.parameters():
    print(param.shape)

result:

Method Four

Using state_dict, state_dict keeps the parameter names and parameters of the model in a dictionary, and all parameters can be obtained by traversal.

for name in model.state_dict():
    print("{:30s}:{}".format(name,model.state_dict()[name].shape))

result:

In pytorch, there are not only models with parameters, but also optimizer and lr_scheduler, so they all have a common attribute state_dict, so if you want to view their parameter names, this method is universal.

View optimizer parameters

Of course, when viewing the parameters of the optimizer, in addition to the fourth method of viewing model parameters, there is also the following method.

#打印optimizer的参数
print("optimizer.param_groups的长度:{}".format(len(optimizer.param_groups)))
for param_group in optimizer.param_groups:
    print(param_group.keys())
    print([type(value) for value in param_group.values()])
    print('查看学习率: ',param_group['lr'])

result:

optimizer.param_groups的长度:1
dict_keys(['params', 'lr', 'betas', 'eps', 'weight_decay', 'amsgrad', 'initial_lr'])
[<class 'list'>, <class 'float'>, <class 'tuple'>, <class 'float'>, <class 'int'>, <class 'bool'>, <class 'float'>]
查看学习率:  0.01

Freeze training

You can selectively freeze certain layers of the model.

for layer in list(model.parameters())[:2]:
    layer.requires_grad=False

Regarding freezing training, you can read my article .