Problems encountered in the study Pytorch

Pytorch Tutorial Address: pytorch Handbook

: Sometimes github page will load does not come out, Github load does not come out of solution processing method Github website css load does not come out of

After modifying host, if not back to normal, is the need to flush the DNS cache, the computer told my hosts file has been modified. Flush the DNS cache method under Windows: enter the command line, enter the command: ipconfig / flushdns

---

1. Pytorch-MNIST dataset Handbook 3.2 Digital Recognition

1.1 Definitions Test section:

pred = output.max (1, keepdim = True) [1] # find the most probable index

torch.max () function --- explanation and examples

a = torch.randn(3,3)
torch.max)(a,0) #返回每一列中最大值的那个元素，且返回索引（返回最大元素在这一列的行索引）
torch.max(a,1) #返回每一行中最大值的那个元素，且返回其索引（返回最大元素在这一行的列索引）

Here is this:

output.max (1, keepdim = True) ---> Returns the largest element in each row and returns the index to return the two arrays

output.max (1, keepdim = True) [1] is to take a second array, taking an indexed array.

1.2 part of the data set

batch_size = 512, the size of the training set of 60,000. Therefore, a total of 60,000 / 512 = 117.18

train_loader = torch.utils.data.DataLoader(
        datasets.MNIST('data', train=True, download=True, 
                       transform=transforms.Compose([
                           transforms.ToTensor(),
                           transforms.Normalize((0.1307,), (0.3081,))
                       ])),
        batch_size=BATCH_SIZE, shuffle=True)
len(train_loader)

>>118

We produced a batch_size parameters train_loader, transform the data into a tensor torch, transforms.Normalize ((0.1307,), (0.3081,)) is the normalized data, the mean and variance based on the data are 0.1307,0.3081 set quite good.

2. Multi-GPU training

2.1 stand-alone multi-GPU torch.nn.DataParalle

用torch.nn.DataParalle We will just our own model as a parameter, you can direct incoming

#使用内置的一个模型，我们这里以resnet50为例
model = torchvision.models.resnet50()
#模型使用多GPU
mdp = torch.nn.DataParallel(model)
mdp

2.2 torch.distributed

2.3 torch.utils.checkpoint