First of all, declare that this is some of the problems and related solutions that I encountered during my study. I sort it out, on the one hand, to record the learning process; on the other hand, to share, maybe help everyone, learn from each other (some of the solutions are unavoidable to refer to others And it is a great honor to sort it out if there are similarities).
1. How to load the downloaded CIFAR10 into python (torchvision.datasets.CIFAR10) || Solve the problem of downloading cifar-10-python.tar.gz directly in python is too slow
Unzip the cifar-10-python.tar.gz downloaded from the https://pan.baidu.com/s/1oAn8o8i link to bring up the cifar-10-batches-py folder, and then place it in the ./data directory (jupyter notebook A sub-directory in) and then set download=False
. Transform=transform in the function is the pre-processing operation defined previously.
2. Code np.transpose(npimg, (1,2,0)) parameter details
def imshow(img):
img = img / 2 + 0.5 # unnormalize
npimg = img.numpy() # 将torch.FloatTensor 转换为numpy
plt.imshow(np.transpose(npimg,(1,2,0)))
plt.show()
The input order of the parameters received by plt.imshow() is (imagesize, imagesize, channels), and the format of the original parameter img is (channels, imagesize, imagessize), call the np.transpose() function to change the input order of the parameters. For example: replace 3×32×32 with 32×32×3.
3. The solution to "jupyter notebook service seems to be down, but it will restart immediately...".
The memory allocation is insufficient. If there is a GPU, the following three lines of program can be introduced at the beginning of the file.
import os
os.environ["CUDA_VISIBLE_DEVICES"] = "0" # 使用第1个GPU
os.environ["KMP_DUPLICATE_LIB_OK"] = 'True' # 允许副本存在
4. About torch.utils.data.DataLoader
torch.utils.data.DataLoader（
dataset，#数据加载
batch_size = 1，#批处理大小设置
shuffle = False，#是否进项洗牌操作
sampler = None，#指定数据加载中使用的索引/键的序列
batch_sampler = None，#和sampler类似
num_workers = 0，#是否进行多进程加载数据设置
collate_fn = None，#是否合并样本列表以形成一小批Tensor
pin_memory = False，#如果True，数据加载器会在返回之前将Tensors复制到CUDA固定内存
drop_last = False，#True如果数据集大小不能被批处理大小整除，则设置为删除最后一个不完整的批处理。
timeout = 0，#如果为正，则为从工作人员收集批处理的超时值
worker_init_fn = None ）
5. Pytorch loads the trained model
Use VGG-16 model
model = models.vgg16(pretrained=False) # 由于是加载的已训练好的模型，此处可以设置为False
pre = torch.load(r'F:\installment\vgg16-397923af.pth') # 提取本地模型
model.load_state_dict(pre)
Load the pre-trained pre-trained model on COCO
model = torchvision.models.detection.fasterrcnn_resnet50_fpn(pretrained=False,pretrained_backbone=False)
model.load_state_dict(torch.load('./model/fasterrcnn_resnet50_fpn_coco-258fb6c6.pth'))
#Load a pre-trained model for classification, only return features
model = torchvision.models.mobilenet_v2(pretrained=False)
model.load_state_dict(torch.load('./model/mobilenet_v2-b0353104.pth'))
backbone = model.features
6. The difference between torch.nn.XXX and torch.functional.XXX
The similarities between the two:
The actual functions of nn.XXX and nn.functional.XXX are the same, that is, nn.Conv2d and nn.functional.conv2d are both convolution, and nn.Dropout and nn.functional.dropout are both Perform dropout. . . . . ;
Operating efficiency is almost the same.
nn.functional.XXX is a functional interface, and nn.Xxx is a class encapsulation of nn.functional.XXX, and nn.Xxx all inherit from a common ancestor nn.Module. This leads to nn.XXX besides having nn.functional.XXX functions, it also has nn.Module related attributes and methods, such as train(), eval(), load_state_dict, state_dict, etc.
The difference between the two:
1. The calling methods of the two are different.
nn.XXX needs to be instantiated and passed in parameters, and then the instantiated object is called in a function call and the input data is passed in.
inputs = torch.rand(64, 3, 244, 244)
conv = nn.Conv2d(in_channels=3, out_channels=64, kernel_size=3, padding=1)
out = conv(inputs)
nn.functional.XXX also passes in input data and other parameters such as weight and bias.
weight = torch.rand(64,3,3,3)
bias = torch.rand(64)
out = nn.functional.conv2d(inputs, weight, bias, padding=1)
2. nn.XXX is inherited from nn.Module and can be used in combination with nn.Sequential, but nn.functional.XXX cannot be used in combination with nn.Sequential.
fm_layer = nn.Sequential(
nn.Conv2d(3, 64, kernel_size=3, padding=1),
nn.BatchNorm2d(num_features=64),
nn.ReLU(),
nn.MaxPool2d(kernel_size=2),
nn.Dropout(0.2)
)
3. nn.XXX does not require you to define and manage the weight yourself; while nn.functional.XXX requires you to define the weight yourself, you need to manually pass in the weight each time you call it, which is not conducive to code reuse.
(1) Use nn.Xxx to define a CNN.
class CNN(nn.Module):
def __init__(self):
super(CNN, self).__init__()
self.cnn1 = nn.Conv2d(in_channels=1, out_channels=16, kernel_size=5,padding=0)
self.relu1 = nn.ReLU()
self.maxpool1 = nn.MaxPool2d(kernel_size=2)
self.cnn2 = nn.Conv2d(in_channels=16, out_channels=32, kernel_size=5, padding=0)
self.relu2 = nn.ReLU() self.maxpool2 = nn.MaxPool2d(kernel_size=2)
self.linear1 = nn.Linear(4 * 4 * 32, 10)
def forward(self, x):
x = x.view(x.size(0), -1)
out = self.maxpool1(self.relu1(self.cnn1(x)))
out = self.maxpool2(self.relu2(self.cnn2(out)))
out = self.linear1(out.view(x.size(0), -1))
return out
(2) Use nn.function.xxx to define the same CNN as above.
class CNN(nn.Module):
def __init__(self):
super(CNN, self).__init__()
self.cnn1_weight = nn.Parameter(torch.rand(16, 1, 5, 5))
self.bias1_weight = nn.Parameter(torch.rand(16))
self.cnn2_weight = nn.Parameter(torch.rand(32, 16, 5, 5))
self.bias2_weight = nn.Parameter(torch.rand(32))
self.linear1_weight = nn.Parameter(torch.rand(4 * 4 * 32, 10))
self.bias3_weight = nn.Parameter(torch.rand(10))
def forward(self, x):
x = x.view(x.size(0), -1)
out = F.conv2d(x, self.cnn1_weight, self.bias1_weight)
out = F.relu(out) out = F.max_pool2d(out)
out = F.conv2d(x, self.cnn2_weight, self.bias2_weight)
out = F.relu(out)
out = F.max_pool2d(out)
out = F.linear(x, self.linear1_weight, self.bias3_weight)
return out
PyTorch official recommendation: Use nn.Xxx for learning parameters (for example, conv2d, linear, batch_norm), and use nn.functional.xxx or nn for those without learning parameters (for example, maxpool, loss func, activation func), etc. .Xxx way. But regarding dropout, I strongly recommend using the nn.Xxx method, because in general, dropout is only performed during the training phase, and no dropout is performed during the eval phase. Use nn.Xxx to define dropout. After calling model.eval(), all the dropout layers in the model are closed, but define dropout in nn.function.dropout way. Dropout cannot be closed after calling model.eval().
(Source Know-It’s better to have sugar)
7. Initialize model parameters
The module parameters of nn.Module in PyTorch all adopt a more reasonable initialization strategy (refer to the source code for the specific initialization method sampled by different types of layers). But it is often necessary to use other methods to initialize the weights. PyTorch's init module provides a variety of preset initialization methods.
#将权重参数初始化成均值为0、标准差为0.01的正态分布随机数，并依然将偏差参数清零。
for name, param in net.named_parameters():
if 'weight' in name:
init.normal_(param, mean=0, std=0.01)
print(name, param.data)
for name, param in net.named_parameters():
if 'bias' in name:
init.constant_(param, val=0)
print(name, param.data)
8. Dimensionality reduction
# numpy中的ravel()、flatten()、squeeze()
# 都有将多维数组转换为一维数组的功能，区别：
# ravel()：如果没有必要，不会产生源数据的副本
# flatten()：返回源数据的副本
# squeeze()：只能对维数为1的维度降维
9. Choice() in numpy
a1 = np.random.choice(a=5, size=3, replace=False, p=None)
# 参数意思分别 是从a 中以概率P，随机选择3个, p没有指定的时候相当于是一致的分布
# replacement 代表的意思是抽样之后还放不放回去，如果是False的话，那么出来的三个数都不一样，如果是
# True的话， 有可能会出现重复的，因为前面的抽的放回去了