一、ResNet18模型设计
ResNet18网络采用了残差模块的跳跃连接思想，缓解了在深度神经网络中增加深度带来的梯度消失问题。
残差构建模块封装成类，代码如下：
class ResBlock(torch.nn.Module):
def __init__(self, ch_in, ch_out, stride=1):
super(ResBlock, self).__init__()
self.conv1 = torch.nn.Conv2d(ch_in,ch_out,kernel_size=3,stride=stride,padding=1)
self.bn1 = torch.nn.BatchNorm2d(ch_out)
self.conv2 = torch.nn.Conv2d(ch_out,ch_out,kernel_size=3,stride=1,padding=1)
self.bn2 = torch.nn.BatchNorm2d(ch_out)
self.extra = torch.nn.Sequential()
if ch_in != ch_out:
# [b,ch_in,h,w] => [b,ch_out,h,w]
self.extra = torch.nn.Sequential(
torch.nn.Conv2d(ch_in,ch_out,kernel_size=1,stride=stride),
torch.nn.BatchNorm2d(ch_out)
)
def forward(self, x):
y = F.relu(self.bn1(self.conv1(x)))
y = self.bn2(self.conv2(y))
# short cut
# extra module: [b,ch_in,h,w] => [b,ch_out,h,w]
y = self.extra(x) + y
return y
嵌入残差模块的ResNet18网络模型代码如下：
class ResNet18(torch.nn.Module):
def __init__(self):
super(ResNet18, self).__init__()
self.conv = torch.nn.Sequential(
torch.nn.Conv2d(3,64,kernel_size=3,stride=3),
torch.nn.BatchNorm2d(64)
)
# 接4个blocks
# [b,64,h,w] => [b,128,h,w]
self.blk1 = ResBlock(64,128,stride=2)
# [b,128,h,w] => [b,256,h,w]
self.blk2 = ResBlock(128,256,stride=2)
# [b,256,h,w] => [b,512,h,w]
self.blk3 = ResBlock(256,512,stride=2)
# [b,512,h,w] => [b,1024,h,w]
self.blk4 = ResBlock(512,512,stride=2)
self.outlayer = torch.nn.Linear(512,10)
def forward(self,x):
x = F.relu(self.conv(x))
# [b,64,h,w] => [b,512,h,w]
x = self.blk1(x)
x = self.blk2(x)
x = self.blk3(x)
x = self.blk4(x)
# [b,512,h,w] => [b,512,1,1]
x = F.adaptive_avg_pool2d(x, [1, 1])
x = x.view(x.size(0),-1)
x = self.outlayer(x)
return x
将该模型用于CIFAR-10数据集的分类问题上，运行结果为：
Training for 100 epochs...
[0m 13s] Epoch 1 [9536/50000] loss = 2.05
[0m 23s] Epoch 1 [19136/50000] loss = 1.65
[0m 34s] Epoch 1 [28736/50000] loss = 1.55
[0m 44s] Epoch 1 [38336/50000] loss = 1.47
[0m 54s] Epoch 1 [47936/50000] loss = 1.41
Accuracy on test set: 51.01% [5101/10000]
[1m 10s] Epoch 2 [9536/50000] loss = 1.32
[1m 20s] Epoch 2 [19136/50000] loss = 1.27
[1m 31s] Epoch 2 [28736/50000] loss = 1.26
[1m 41s] Epoch 2 [38336/50000] loss = 1.19
[1m 52s] Epoch 2 [47936/50000] loss = 1.15
Accuracy on test set: 59.43% [5943/10000]
[2m 7s] Epoch 3 [9536/50000] loss = 1.09
[2m 18s] Epoch 3 [19136/50000] loss = 1.04
[2m 28s] Epoch 3 [28736/50000] loss = 1.04
[2m 38s] Epoch 3 [38336/50000] loss = 1.02
[2m 49s] Epoch 3 [47936/50000] loss = 1.01
Accuracy on test set: 60.87% [6087/10000]
[3m 5s] Epoch 4 [9536/50000] loss = 0.90
[3m 15s] Epoch 4 [19136/50000] loss = 0.91
[3m 25s] Epoch 4 [28736/50000] loss = 0.91
[3m 35s] Epoch 4 [38336/50000] loss = 0.90
[3m 46s] Epoch 4 [47936/50000] loss = 0.89
Accuracy on test set: 66.56% [6656/10000]
[4m 2s] Epoch 5 [9536/50000] loss = 0.79
[4m 14s] Epoch 5 [19136/50000] loss = 0.81
[4m 26s] Epoch 5 [28736/50000] loss = 0.78
[4m 37s] Epoch 5 [38336/50000] loss = 0.81
[4m 48s] Epoch 5 [47936/50000] loss = 0.81
Accuracy on test set: 67.69% [6769/10000]
[5m 6s] Epoch 6 [9536/50000] loss = 0.66
[5m 16s] Epoch 6 [19136/50000] loss = 0.73
[5m 27s] Epoch 6 [28736/50000] loss = 0.73
[5m 38s] Epoch 6 [38336/50000] loss = 0.72
[5m 48s] Epoch 6 [47936/50000] loss = 0.72
Accuracy on test set: 69.18% [6918/10000]
[6m 5s] Epoch 7 [9536/50000] loss = 0.61
[6m 15s] Epoch 7 [19136/50000] loss = 0.63
[6m 26s] Epoch 7 [28736/50000] loss = 0.63
[6m 37s] Epoch 7 [38336/50000] loss = 0.65
[6m 47s] Epoch 7 [47936/50000] loss = 0.67
Accuracy on test set: 69.4% [6940/10000]
[7m 3s] Epoch 8 [9536/50000] loss = 0.52
[7m 14s] Epoch 8 [19136/50000] loss = 0.56
[7m 25s] Epoch 8 [28736/50000] loss = 0.57
[7m 35s] Epoch 8 [38336/50000] loss = 0.57
[7m 46s] Epoch 8 [47936/50000] loss = 0.57
Accuracy on test set: 69.53% [6953/10000]
[8m 2s] Epoch 9 [9536/50000] loss = 0.45
[8m 12s] Epoch 9 [19136/50000] loss = 0.47
[8m 23s] Epoch 9 [28736/50000] loss = 0.51
[8m 33s] Epoch 9 [38336/50000] loss = 0.50
[8m 44s] Epoch 9 [47936/50000] loss = 0.51
Accuracy on test set: 71.12% [7112/10000]
[9m 0s] Epoch 10 [9536/50000] loss = 0.35
[9m 10s] Epoch 10 [19136/50000] loss = 0.40
[9m 21s] Epoch 10 [28736/50000] loss = 0.45
[9m 31s] Epoch 10 [38336/50000] loss = 0.44
[9m 42s] Epoch 10 [47936/50000] loss = 0.43
Accuracy on test set: 70.43% [7043/10000]
[9m 59s] Epoch 11 [9536/50000] loss = 0.29
[10m 9s] Epoch 11 [19136/50000] loss = 0.35
[10m 20s] Epoch 11 [28736/50000] loss = 0.39
[10m 30s] Epoch 11 [38336/50000] loss = 0.37
[10m 41s] Epoch 11 [47936/50000] loss = 0.42
Accuracy on test set: 70.64% [7064/10000]
[10m 57s] Epoch 12 [9536/50000] loss = 0.24
[11m 7s] Epoch 12 [19136/50000] loss = 0.28
[11m 18s] Epoch 12 [28736/50000] loss = 0.30
[11m 28s] Epoch 12 [38336/50000] loss = 0.32
[11m 39s] Epoch 12 [47936/50000] loss = 0.34
Accuracy on test set: 70.23% [7023/10000]
[11m 55s] Epoch 13 [9536/50000] loss = 0.21
[12m 6s] Epoch 13 [19136/50000] loss = 0.26
[12m 16s] Epoch 13 [28736/50000] loss = 0.29
[12m 27s] Epoch 13 [38336/50000] loss = 0.30
[12m 37s] Epoch 13 [47936/50000] loss = 0.30
Accuracy on test set: 69.81% [6981/10000]
[12m 53s] Epoch 14 [9536/50000] loss = 0.19
[13m 3s] Epoch 14 [19136/50000] loss = 0.22
[13m 14s] Epoch 14 [28736/50000] loss = 0.22
[13m 25s] Epoch 14 [38336/50000] loss = 0.24
[13m 35s] Epoch 14 [47936/50000] loss = 0.28
Accuracy on test set: 69.29% [6929/10000]
[13m 51s] Epoch 15 [9536/50000] loss = 0.16
[14m 2s] Epoch 15 [19136/50000] loss = 0.17
[14m 13s] Epoch 15 [28736/50000] loss = 0.23
[14m 23s] Epoch 15 [38336/50000] loss = 0.23
[14m 34s] Epoch 15 [47936/50000] loss = 0.22
Accuracy on test set: 69.57% [6957/10000]
[14m 50s] Epoch 16 [9536/50000] loss = 0.17
[15m 1s] Epoch 16 [19136/50000] loss = 0.17
[15m 11s] Epoch 16 [28736/50000] loss = 0.18
[15m 22s] Epoch 16 [38336/50000] loss = 0.22
[15m 33s] Epoch 16 [47936/50000] loss = 0.21
Accuracy on test set: 69.93% [6993/10000]
[15m 49s] Epoch 17 [9536/50000] loss = 0.14
[15m 59s] Epoch 17 [19136/50000] loss = 0.17
[16m 10s] Epoch 17 [28736/50000] loss = 0.20
[16m 20s] Epoch 17 [38336/50000] loss = 0.17
[16m 30s] Epoch 17 [47936/50000] loss = 0.21
Accuracy on test set: 69.48% [6948/10000]
[16m 47s] Epoch 18 [9536/50000] loss = 0.11
[16m 58s] Epoch 18 [19136/50000] loss = 0.13
[17m 8s] Epoch 18 [28736/50000] loss = 0.15
[17m 19s] Epoch 18 [38336/50000] loss = 0.17
[17m 29s] Epoch 18 [47936/50000] loss = 0.16
Accuracy on test set: 70.04% [7004/10000]
[17m 45s] Epoch 19 [9536/50000] loss = 0.10
[17m 56s] Epoch 19 [19136/50000] loss = 0.14
[18m 6s] Epoch 19 [28736/50000] loss = 0.14
[18m 17s] Epoch 19 [38336/50000] loss = 0.16
[18m 27s] Epoch 19 [47936/50000] loss = 0.17
Accuracy on test set: 70.04% [7004/10000]
[18m 43s] Epoch 20 [9536/50000] loss = 0.10
[18m 54s] Epoch 20 [19136/50000] loss = 0.11
[19m 4s] Epoch 20 [28736/50000] loss = 0.14
[19m 15s] Epoch 20 [38336/50000] loss = 0.15
[19m 25s] Epoch 20 [47936/50000] loss = 0.17
Accuracy on test set: 69.41% [6941/10000]
[19m 42s] Epoch 21 [9536/50000] loss = 0.10
[19m 52s] Epoch 21 [19136/50000] loss = 0.12
[20m 3s] Epoch 21 [28736/50000] loss = 0.12
[20m 13s] Epoch 21 [38336/50000] loss = 0.14
[20m 24s] Epoch 21 [47936/50000] loss = 0.15
Accuracy on test set: 70.5% [7050/10000]
[20m 41s] Epoch 22 [9536/50000] loss = 0.11
[20m 51s] Epoch 22 [19136/50000] loss = 0.11
[21m 2s] Epoch 22 [28736/50000] loss = 0.13
[21m 12s] Epoch 22 [38336/50000] loss = 0.13
[21m 22s] Epoch 22 [47936/50000] loss = 0.16
Accuracy on test set: 70.63% [7063/10000]
[21m 39s] Epoch 23 [9536/50000] loss = 0.09
[21m 50s] Epoch 23 [19136/50000] loss = 0.10
[22m 0s] Epoch 23 [28736/50000] loss = 0.11
[22m 11s] Epoch 23 [38336/50000] loss = 0.13
[22m 21s] Epoch 23 [47936/50000] loss = 0.12
Accuracy on test set: 70.15% [7015/10000]
[22m 37s] Epoch 24 [9536/50000] loss = 0.08
[22m 48s] Epoch 24 [19136/50000] loss = 0.08
[22m 58s] Epoch 24 [28736/50000] loss = 0.11
[23m 9s] Epoch 24 [38336/50000] loss = 0.12
[23m 19s] Epoch 24 [47936/50000] loss = 0.15
Accuracy on test set: 69.06% [6906/10000]
[23m 34s] Epoch 25 [9536/50000] loss = 0.12
[23m 44s] Epoch 25 [19136/50000] loss = 0.09
[23m 55s] Epoch 25 [28736/50000] loss = 0.11
[24m 5s] Epoch 25 [38336/50000] loss = 0.11
[24m 16s] Epoch 25 [47936/50000] loss = 0.12
Accuracy on test set: 70.13% [7013/10000]
[24m 33s] Epoch 26 [9536/50000] loss = 0.06
[24m 43s] Epoch 26 [19136/50000] loss = 0.06
[24m 54s] Epoch 26 [28736/50000] loss = 0.10
[25m 4s] Epoch 26 [38336/50000] loss = 0.12
[25m 15s] Epoch 26 [47936/50000] loss = 0.12
Accuracy on test set: 70.43% [7043/10000]
[25m 31s] Epoch 27 [9536/50000] loss = 0.08
[25m 41s] Epoch 27 [19136/50000] loss = 0.08
[25m 52s] Epoch 27 [28736/50000] loss = 0.11
[26m 2s] Epoch 27 [38336/50000] loss = 0.10
[26m 12s] Epoch 27 [47936/50000] loss = 0.12
Accuracy on test set: 69.32% [6932/10000]
[26m 28s] Epoch 28 [9536/50000] loss = 0.09
[26m 39s] Epoch 28 [19136/50000] loss = 0.08
[26m 50s] Epoch 28 [28736/50000] loss = 0.08
[27m 0s] Epoch 28 [38336/50000] loss = 0.08
[27m 11s] Epoch 28 [47936/50000] loss = 0.11
Accuracy on test set: 69.49% [6949/10000]
[27m 27s] Epoch 29 [9536/50000] loss = 0.08
[27m 37s] Epoch 29 [19136/50000] loss = 0.07
[27m 48s] Epoch 29 [28736/50000] loss = 0.09
[27m 59s] Epoch 29 [38336/50000] loss = 0.08
[28m 9s] Epoch 29 [47936/50000] loss = 0.11
Accuracy on test set: 70.13% [7013/10000]
[28m 25s] Epoch 30 [9536/50000] loss = 0.11
[28m 35s] Epoch 30 [19136/50000] loss = 0.08
[28m 46s] Epoch 30 [28736/50000] loss = 0.08
[28m 56s] Epoch 30 [38336/50000] loss = 0.09
[29m 7s] Epoch 30 [47936/50000] loss = 0.10
Accuracy on test set: 70.34% [7034/10000]
[29m 23s] Epoch 31 [9536/50000] loss = 0.06
[29m 33s] Epoch 31 [19136/50000] loss = 0.07
[29m 44s] Epoch 31 [28736/50000] loss = 0.07
[29m 54s] Epoch 31 [38336/50000] loss = 0.08
[30m 5s] Epoch 31 [47936/50000] loss = 0.10
Accuracy on test set: 69.94% [6994/10000]
[30m 20s] Epoch 32 [9536/50000] loss = 0.08
[30m 31s] Epoch 32 [19136/50000] loss = 0.07
[30m 41s] Epoch 32 [28736/50000] loss = 0.08
[30m 52s] Epoch 32 [38336/50000] loss = 0.09
[31m 2s] Epoch 32 [47936/50000] loss = 0.08
Accuracy on test set: 70.89% [7089/10000]
[31m 18s] Epoch 33 [9536/50000] loss = 0.08
[31m 29s] Epoch 33 [19136/50000] loss = 0.06
[31m 39s] Epoch 33 [28736/50000] loss = 0.06
[31m 50s] Epoch 33 [38336/50000] loss = 0.06
[32m 0s] Epoch 33 [47936/50000] loss = 0.10
Accuracy on test set: 71.08% [7108/10000]
[32m 17s] Epoch 34 [9536/50000] loss = 0.07
[32m 27s] Epoch 34 [19136/50000] loss = 0.07
[32m 39s] Epoch 34 [28736/50000] loss = 0.06
[32m 50s] Epoch 34 [38336/50000] loss = 0.08
[33m 0s] Epoch 34 [47936/50000] loss = 0.09
Accuracy on test set: 70.19% [7019/10000]
[33m 16s] Epoch 35 [9536/50000] loss = 0.11
[33m 27s] Epoch 35 [19136/50000] loss = 0.09
[33m 38s] Epoch 35 [28736/50000] loss = 0.07
[33m 48s] Epoch 35 [38336/50000] loss = 0.08
[33m 59s] Epoch 35 [47936/50000] loss = 0.07
Accuracy on test set: 70.29% [7029/10000]
[34m 15s] Epoch 36 [9536/50000] loss = 0.05
[34m 26s] Epoch 36 [19136/50000] loss = 0.05
[34m 36s] Epoch 36 [28736/50000] loss = 0.05
[34m 47s] Epoch 36 [38336/50000] loss = 0.06
[34m 57s] Epoch 36 [47936/50000] loss = 0.07
Accuracy on test set: 69.83% [6983/10000]
[35m 14s] Epoch 37 [9536/50000] loss = 0.07
[35m 24s] Epoch 37 [19136/50000] loss = 0.07
[35m 35s] Epoch 37 [28736/50000] loss = 0.06
[35m 45s] Epoch 37 [38336/50000] loss = 0.06
[35m 56s] Epoch 37 [47936/50000] loss = 0.09
Accuracy on test set: 69.18% [6918/10000]
[36m 12s] Epoch 38 [9536/50000] loss = 0.08
[36m 23s] Epoch 38 [19136/50000] loss = 0.08
[36m 33s] Epoch 38 [28736/50000] loss = 0.07
[36m 44s] Epoch 38 [38336/50000] loss = 0.06
[36m 54s] Epoch 38 [47936/50000] loss = 0.08
Accuracy on test set: 70.81% [7081/10000]
[37m 9s] Epoch 39 [9536/50000] loss = 0.04
[37m 19s] Epoch 39 [19136/50000] loss = 0.06
[37m 29s] Epoch 39 [28736/50000] loss = 0.07
[37m 39s] Epoch 39 [38336/50000] loss = 0.06
[37m 49s] Epoch 39 [47936/50000] loss = 0.06
Accuracy on test set: 69.18% [6918/10000]
[38m 4s] Epoch 40 [9536/50000] loss = 0.04
[38m 14s] Epoch 40 [19136/50000] loss = 0.06
[38m 24s] Epoch 40 [28736/50000] loss = 0.06
[38m 35s] Epoch 40 [38336/50000] loss = 0.07
[38m 45s] Epoch 40 [47936/50000] loss = 0.11
Accuracy on test set: 69.37% [6937/10000]
[39m 0s] Epoch 41 [9536/50000] loss = 0.07
[39m 10s] Epoch 41 [19136/50000] loss = 0.05
[39m 20s] Epoch 41 [28736/50000] loss = 0.06
[39m 30s] Epoch 41 [38336/50000] loss = 0.08
[39m 40s] Epoch 41 [47936/50000] loss = 0.08
Accuracy on test set: 71.13% [7113/10000]
[39m 55s] Epoch 42 [9536/50000] loss = 0.05
[40m 5s] Epoch 42 [19136/50000] loss = 0.05
[40m 15s] Epoch 42 [28736/50000] loss = 0.04
[40m 25s] Epoch 42 [38336/50000] loss = 0.06
[40m 35s] Epoch 42 [47936/50000] loss = 0.07
Accuracy on test set: 70.55% [7055/10000]
[40m 51s] Epoch 43 [9536/50000] loss = 0.05
[41m 1s] Epoch 43 [19136/50000] loss = 0.05
[41m 11s] Epoch 43 [28736/50000] loss = 0.05
[41m 22s] Epoch 43 [38336/50000] loss = 0.07
[41m 32s] Epoch 43 [47936/50000] loss = 0.08
Accuracy on test set: 70.33% [7033/10000]
[41m 47s] Epoch 44 [9536/50000] loss = 0.05
[41m 57s] Epoch 44 [19136/50000] loss = 0.06
[42m 7s] Epoch 44 [28736/50000] loss = 0.05
[42m 17s] Epoch 44 [38336/50000] loss = 0.06
[42m 27s] Epoch 44 [47936/50000] loss = 0.06
Accuracy on test set: 70.25% [7025/10000]
[42m 42s] Epoch 45 [9536/50000] loss = 0.05
[42m 53s] Epoch 45 [19136/50000] loss = 0.05
[43m 3s] Epoch 45 [28736/50000] loss = 0.05
[43m 14s] Epoch 45 [38336/50000] loss = 0.06
[43m 24s] Epoch 45 [47936/50000] loss = 0.06
Accuracy on test set: 70.27% [7027/10000]
[43m 40s] Epoch 46 [9536/50000] loss = 0.04
[43m 50s] Epoch 46 [19136/50000] loss = 0.05
[44m 0s] Epoch 46 [28736/50000] loss = 0.08
[44m 11s] Epoch 46 [38336/50000] loss = 0.09
[44m 21s] Epoch 46 [47936/50000] loss = 0.07
Accuracy on test set: 70.83% [7083/10000]
[44m 37s] Epoch 47 [9536/50000] loss = 0.06
[44m 47s] Epoch 47 [19136/50000] loss = 0.05
[44m 57s] Epoch 47 [28736/50000] loss = 0.05
[45m 7s] Epoch 47 [38336/50000] loss = 0.04
[45m 18s] Epoch 47 [47936/50000] loss = 0.04
Accuracy on test set: 71.54% [7154/10000]
[45m 33s] Epoch 48 [9536/50000] loss = 0.03
[45m 44s] Epoch 48 [19136/50000] loss = 0.04
[45m 54s] Epoch 48 [28736/50000] loss = 0.05
[46m 4s] Epoch 48 [38336/50000] loss = 0.07
[46m 14s] Epoch 48 [47936/50000] loss = 0.10
Accuracy on test set: 69.47% [6947/10000]
[46m 30s] Epoch 49 [9536/50000] loss = 0.08
[46m 40s] Epoch 49 [19136/50000] loss = 0.06
[46m 51s] Epoch 49 [28736/50000] loss = 0.05
[47m 1s] Epoch 49 [38336/50000] loss = 0.05
[47m 11s] Epoch 49 [47936/50000] loss = 0.04
Accuracy on test set: 70.7% [7070/10000]
[47m 26s] Epoch 50 [9536/50000] loss = 0.03
[47m 36s] Epoch 50 [19136/50000] loss = 0.03
[47m 47s] Epoch 50 [28736/50000] loss = 0.05
[47m 57s] Epoch 50 [38336/50000] loss = 0.05
[48m 7s] Epoch 50 [47936/50000] loss = 0.05
Accuracy on test set: 70.87% [7087/10000]
[48m 23s] Epoch 51 [9536/50000] loss = 0.05
[48m 33s] Epoch 51 [19136/50000] loss = 0.07
[48m 43s] Epoch 51 [28736/50000] loss = 0.05
[48m 53s] Epoch 51 [38336/50000] loss = 0.06
[49m 4s] Epoch 51 [47936/50000] loss = 0.06
Accuracy on test set: 70.72% [7072/10000]
[49m 19s] Epoch 52 [9536/50000] loss = 0.04
[49m 29s] Epoch 52 [19136/50000] loss = 0.04
[49m 40s] Epoch 52 [28736/50000] loss = 0.04
[49m 50s] Epoch 52 [38336/50000] loss = 0.05
[50m 0s] Epoch 52 [47936/50000] loss = 0.08
Accuracy on test set: 69.14% [6914/10000]
[50m 15s] Epoch 53 [9536/50000] loss = 0.12
[50m 26s] Epoch 53 [19136/50000] loss = 0.07
[50m 36s] Epoch 53 [28736/50000] loss = 0.05
[50m 46s] Epoch 53 [38336/50000] loss = 0.04
[50m 56s] Epoch 53 [47936/50000] loss = 0.04
Accuracy on test set: 71.32% [7132/10000]
[51m 12s] Epoch 54 [9536/50000] loss = 0.02
[51m 22s] Epoch 54 [19136/50000] loss = 0.03
[51m 32s] Epoch 54 [28736/50000] loss = 0.03
[51m 42s] Epoch 54 [38336/50000] loss = 0.04
[51m 52s] Epoch 54 [47936/50000] loss = 0.04
Accuracy on test set: 69.72% [6972/10000]
[52m 8s] Epoch 55 [9536/50000] loss = 0.04
[52m 19s] Epoch 55 [19136/50000] loss = 0.05
[52m 29s] Epoch 55 [28736/50000] loss = 0.06
[52m 39s] Epoch 55 [38336/50000] loss = 0.05
[52m 49s] Epoch 55 [47936/50000] loss = 0.05
Accuracy on test set: 70.32% [7032/10000]
[53m 5s] Epoch 56 [9536/50000] loss = 0.05
[53m 15s] Epoch 56 [19136/50000] loss = 0.04
[53m 25s] Epoch 56 [28736/50000] loss = 0.04
[53m 36s] Epoch 56 [38336/50000] loss = 0.04
[53m 46s] Epoch 56 [47936/50000] loss = 0.05
Accuracy on test set: 70.13% [7013/10000]
[54m 2s] Epoch 57 [9536/50000] loss = 0.02
[54m 12s] Epoch 57 [19136/50000] loss = 0.03
[54m 22s] Epoch 57 [28736/50000] loss = 0.02
[54m 32s] Epoch 57 [38336/50000] loss = 0.05
[54m 42s] Epoch 57 [47936/50000] loss = 0.05
Accuracy on test set: 69.87% [6987/10000]
[54m 58s] Epoch 58 [9536/50000] loss = 0.06
[55m 8s] Epoch 58 [19136/50000] loss = 0.05
[55m 18s] Epoch 58 [28736/50000] loss = 0.05
[55m 28s] Epoch 58 [38336/50000] loss = 0.05
[55m 39s] Epoch 58 [47936/50000] loss = 0.04
Accuracy on test set: 69.23% [6923/10000]
[55m 54s] Epoch 59 [9536/50000] loss = 0.11
[56m 4s] Epoch 59 [19136/50000] loss = 0.07
[56m 14s] Epoch 59 [28736/50000] loss = 0.05
[56m 25s] Epoch 59 [38336/50000] loss = 0.05
[56m 35s] Epoch 59 [47936/50000] loss = 0.04
Accuracy on test set: 71.21% [7121/10000]
[56m 50s] Epoch 60 [9536/50000] loss = 0.02
[57m 1s] Epoch 60 [19136/50000] loss = 0.02
[57m 11s] Epoch 60 [28736/50000] loss = 0.03
[57m 21s] Epoch 60 [38336/50000] loss = 0.03
[57m 31s] Epoch 60 [47936/50000] loss = 0.05
Accuracy on test set: 70.63% [7063/10000]
[57m 47s] Epoch 61 [9536/50000] loss = 0.03
[57m 57s] Epoch 61 [19136/50000] loss = 0.04
[58m 7s] Epoch 61 [28736/50000] loss = 0.05
[58m 18s] Epoch 61 [38336/50000] loss = 0.04
[58m 28s] Epoch 61 [47936/50000] loss = 0.05
Accuracy on test set: 70.59% [7059/10000]
[58m 44s] Epoch 62 [9536/50000] loss = 0.04
[58m 54s] Epoch 62 [19136/50000] loss = 0.04
[59m 4s] Epoch 62 [28736/50000] loss = 0.04
[59m 14s] Epoch 62 [38336/50000] loss = 0.03
[59m 24s] Epoch 62 [47936/50000] loss = 0.04
Accuracy on test set: 71.01% [7101/10000]
[59m 40s] Epoch 63 [9536/50000] loss = 0.03
[59m 50s] Epoch 63 [19136/50000] loss = 0.04
[60m 0s] Epoch 63 [28736/50000] loss = 0.05
[60m 11s] Epoch 63 [38336/50000] loss = 0.05
[60m 21s] Epoch 63 [47936/50000] loss = 0.04
Accuracy on test set: 70.1% [7010/10000]
[60m 36s] Epoch 64 [9536/50000] loss = 0.04
[60m 46s] Epoch 64 [19136/50000] loss = 0.05
[60m 57s] Epoch 64 [28736/50000] loss = 0.05
[61m 7s] Epoch 64 [38336/50000] loss = 0.04
[61m 17s] Epoch 64 [47936/50000] loss = 0.04
Accuracy on test set: 70.59% [7059/10000]
[61m 33s] Epoch 65 [9536/50000] loss = 0.03
[61m 43s] Epoch 65 [19136/50000] loss = 0.04
[61m 53s] Epoch 65 [28736/50000] loss = 0.03
[62m 3s] Epoch 65 [38336/50000] loss = 0.05
[62m 13s] Epoch 65 [47936/50000] loss = 0.03
Accuracy on test set: 70.88% [7088/10000]
[62m 29s] Epoch 66 [9536/50000] loss = 0.03
[62m 39s] Epoch 66 [19136/50000] loss = 0.05
[62m 49s] Epoch 66 [28736/50000] loss = 0.04
[62m 59s] Epoch 66 [38336/50000] loss = 0.04
[63m 10s] Epoch 66 [47936/50000] loss = 0.05
Accuracy on test set: 70.89% [7089/10000]
[63m 25s] Epoch 67 [9536/50000] loss = 0.02
[63m 35s] Epoch 67 [19136/50000] loss = 0.02
[63m 46s] Epoch 67 [28736/50000] loss = 0.04
[63m 56s] Epoch 67 [38336/50000] loss = 0.05
[64m 6s] Epoch 67 [47936/50000] loss = 0.06
Accuracy on test set: 69.52% [6952/10000]
[64m 22s] Epoch 68 [9536/50000] loss = 0.07
[64m 32s] Epoch 68 [19136/50000] loss = 0.04
[64m 42s] Epoch 68 [28736/50000] loss = 0.04
[64m 53s] Epoch 68 [38336/50000] loss = 0.03
[65m 3s] Epoch 68 [47936/50000] loss = 0.04
Accuracy on test set: 70.64% [7064/10000]
[65m 19s] Epoch 69 [9536/50000] loss = 0.02
[65m 29s] Epoch 69 [19136/50000] loss = 0.02
[65m 39s] Epoch 69 [28736/50000] loss = 0.03
[65m 49s] Epoch 69 [38336/50000] loss = 0.04
[66m 0s] Epoch 69 [47936/50000] loss = 0.04
Accuracy on test set: 71.13% [7113/10000]
[66m 15s] Epoch 70 [9536/50000] loss = 0.05
[66m 26s] Epoch 70 [19136/50000] loss = 0.03
[66m 36s] Epoch 70 [28736/50000] loss = 0.03
[66m 46s] Epoch 70 [38336/50000] loss = 0.04
[66m 56s] Epoch 70 [47936/50000] loss = 0.06
Accuracy on test set: 70.67% [7067/10000]
[67m 12s] Epoch 71 [9536/50000] loss = 0.05
[67m 22s] Epoch 71 [19136/50000] loss = 0.04
[67m 32s] Epoch 71 [28736/50000] loss = 0.03
[67m 42s] Epoch 71 [38336/50000] loss = 0.03
[67m 53s] Epoch 71 [47936/50000] loss = 0.02
Accuracy on test set: 71.18% [7118/10000]
[68m 8s] Epoch 72 [9536/50000] loss = 0.02
[68m 18s] Epoch 72 [19136/50000] loss = 0.04
[68m 28s] Epoch 72 [28736/50000] loss = 0.04
[68m 39s] Epoch 72 [38336/50000] loss = 0.04
[68m 49s] Epoch 72 [47936/50000] loss = 0.05
Accuracy on test set: 70.1% [7010/10000]
[69m 4s] Epoch 73 [9536/50000] loss = 0.04
[69m 15s] Epoch 73 [19136/50000] loss = 0.04
[69m 25s] Epoch 73 [28736/50000] loss = 0.04
[69m 35s] Epoch 73 [38336/50000] loss = 0.05
[69m 46s] Epoch 73 [47936/50000] loss = 0.05
Accuracy on test set: 69.98% [6998/10000]
[70m 1s] Epoch 74 [9536/50000] loss = 0.03
[70m 11s] Epoch 74 [19136/50000] loss = 0.03
[70m 22s] Epoch 74 [28736/50000] loss = 0.03
[70m 32s] Epoch 74 [38336/50000] loss = 0.04
[70m 42s] Epoch 74 [47936/50000] loss = 0.05
Accuracy on test set: 70.87% [7087/10000]
[70m 57s] Epoch 75 [9536/50000] loss = 0.04
[71m 8s] Epoch 75 [19136/50000] loss = 0.03
[71m 18s] Epoch 75 [28736/50000] loss = 0.03
[71m 28s] Epoch 75 [38336/50000] loss = 0.03
[71m 38s] Epoch 75 [47936/50000] loss = 0.03
Accuracy on test set: 70.38% [7038/10000]
[71m 54s] Epoch 76 [9536/50000] loss = 0.02
[72m 4s] Epoch 76 [19136/50000] loss = 0.03
[72m 14s] Epoch 76 [28736/50000] loss = 0.03
[72m 24s] Epoch 76 [38336/50000] loss = 0.05
[72m 35s] Epoch 76 [47936/50000] loss = 0.04
Accuracy on test set: 70.03% [7003/10000]
[72m 50s] Epoch 77 [9536/50000] loss = 0.04
[73m 0s] Epoch 77 [19136/50000] loss = 0.03
[73m 11s] Epoch 77 [28736/50000] loss = 0.02
[73m 21s] Epoch 77 [38336/50000] loss = 0.04
[73m 31s] Epoch 77 [47936/50000] loss = 0.03
Accuracy on test set: 70.56% [7056/10000]
[73m 47s] Epoch 78 [9536/50000] loss = 0.04
[73m 57s] Epoch 78 [19136/50000] loss = 0.05
[74m 7s] Epoch 78 [28736/50000] loss = 0.03
[74m 17s] Epoch 78 [38336/50000] loss = 0.04
[74m 27s] Epoch 78 [47936/50000] loss = 0.04
Accuracy on test set: 70.95% [7095/10000]
[74m 43s] Epoch 79 [9536/50000] loss = 0.02
[74m 53s] Epoch 79 [19136/50000] loss = 0.03
[75m 4s] Epoch 79 [28736/50000] loss = 0.03
[75m 14s] Epoch 79 [38336/50000] loss = 0.03
[75m 24s] Epoch 79 [47936/50000] loss = 0.03
Accuracy on test set: 71.11% [7111/10000]
[75m 40s] Epoch 80 [9536/50000] loss = 0.03
[75m 50s] Epoch 80 [19136/50000] loss = 0.03
[76m 0s] Epoch 80 [28736/50000] loss = 0.03
[76m 11s] Epoch 80 [38336/50000] loss = 0.03
[76m 21s] Epoch 80 [47936/50000] loss = 0.03
Accuracy on test set: 70.89% [7089/10000]
[76m 36s] Epoch 81 [9536/50000] loss = 0.03
[76m 46s] Epoch 81 [19136/50000] loss = 0.03
[76m 57s] Epoch 81 [28736/50000] loss = 0.04
[77m 7s] Epoch 81 [38336/50000] loss = 0.04
[77m 17s] Epoch 81 [47936/50000] loss = 0.04
Accuracy on test set: 70.45% [7045/10000]
[77m 33s] Epoch 82 [9536/50000] loss = 0.05
[77m 43s] Epoch 82 [19136/50000] loss = 0.04
[77m 53s] Epoch 82 [28736/50000] loss = 0.04
[78m 3s] Epoch 82 [38336/50000] loss = 0.03
[78m 14s] Epoch 82 [47936/50000] loss = 0.03
Accuracy on test set: 70.69% [7069/10000]
[78m 29s] Epoch 83 [9536/50000] loss = 0.03
[78m 39s] Epoch 83 [19136/50000] loss = 0.03
[78m 50s] Epoch 83 [28736/50000] loss = 0.02
[79m 0s] Epoch 83 [38336/50000] loss = 0.02
[79m 10s] Epoch 83 [47936/50000] loss = 0.03
Accuracy on test set: 71.04% [7104/10000]
[79m 26s] Epoch 84 [9536/50000] loss = 0.04
[79m 36s] Epoch 84 [19136/50000] loss = 0.07
[79m 46s] Epoch 84 [28736/50000] loss = 0.08
[79m 56s] Epoch 84 [38336/50000] loss = 0.04
[80m 6s] Epoch 84 [47936/50000] loss = 0.03
Accuracy on test set: 71.72% [7172/10000]
[80m 22s] Epoch 85 [9536/50000] loss = 0.04
[80m 32s] Epoch 85 [19136/50000] loss = 0.03
[80m 42s] Epoch 85 [28736/50000] loss = 0.03
[80m 53s] Epoch 85 [38336/50000] loss = 0.04
[81m 3s] Epoch 85 [47936/50000] loss = 0.03
Accuracy on test set: 71.23% [7123/10000]
[81m 19s] Epoch 86 [9536/50000] loss = 0.03
[81m 29s] Epoch 86 [19136/50000] loss = 0.02
[81m 39s] Epoch 86 [28736/50000] loss = 0.02
[81m 50s] Epoch 86 [38336/50000] loss = 0.02
[82m 0s] Epoch 86 [47936/50000] loss = 0.02
Accuracy on test set: 71.38% [7138/10000]
[82m 16s] Epoch 87 [9536/50000] loss = 0.02
[82m 26s] Epoch 87 [19136/50000] loss = 0.03
[82m 36s] Epoch 87 [28736/50000] loss = 0.03
[82m 47s] Epoch 87 [38336/50000] loss = 0.03
[82m 57s] Epoch 87 [47936/50000] loss = 0.04
Accuracy on test set: 70.79% [7079/10000]
[83m 12s] Epoch 88 [9536/50000] loss = 0.02
[83m 23s] Epoch 88 [19136/50000] loss = 0.02
[83m 33s] Epoch 88 [28736/50000] loss = 0.03
[83m 43s] Epoch 88 [38336/50000] loss = 0.03
[83m 54s] Epoch 88 [47936/50000] loss = 0.08
Accuracy on test set: 69.79% [6979/10000]
[84m 9s] Epoch 89 [9536/50000] loss = 0.05
[84m 20s] Epoch 89 [19136/50000] loss = 0.04
[84m 30s] Epoch 89 [28736/50000] loss = 0.03
[84m 40s] Epoch 89 [38336/50000] loss = 0.03
[84m 51s] Epoch 89 [47936/50000] loss = 0.03
Accuracy on test set: 71.09% [7109/10000]
[85m 6s] Epoch 90 [9536/50000] loss = 0.02
[85m 17s] Epoch 90 [19136/50000] loss = 0.01
[85m 27s] Epoch 90 [28736/50000] loss = 0.01
[85m 37s] Epoch 90 [38336/50000] loss = 0.02
[85m 47s] Epoch 90 [47936/50000] loss = 0.02
Accuracy on test set: 70.74% [7074/10000]
[86m 3s] Epoch 91 [9536/50000] loss = 0.04
[86m 14s] Epoch 91 [19136/50000] loss = 0.04
[86m 24s] Epoch 91 [28736/50000] loss = 0.04
[86m 34s] Epoch 91 [38336/50000] loss = 0.03
[86m 44s] Epoch 91 [47936/50000] loss = 0.04
Accuracy on test set: 70.92% [7092/10000]
[87m 0s] Epoch 92 [9536/50000] loss = 0.02
[87m 10s] Epoch 92 [19136/50000] loss = 0.03
[87m 21s] Epoch 92 [28736/50000] loss = 0.03
[87m 31s] Epoch 92 [38336/50000] loss = 0.03
[87m 41s] Epoch 92 [47936/50000] loss = 0.03
Accuracy on test set: 70.67% [7067/10000]
[87m 56s] Epoch 93 [9536/50000] loss = 0.03
[88m 6s] Epoch 93 [19136/50000] loss = 0.03
[88m 16s] Epoch 93 [28736/50000] loss = 0.05
[88m 26s] Epoch 93 [38336/50000] loss = 0.04
[88m 36s] Epoch 93 [47936/50000] loss = 0.05
Accuracy on test set: 70.99% [7099/10000]
[88m 52s] Epoch 94 [9536/50000] loss = 0.04
[89m 2s] Epoch 94 [19136/50000] loss = 0.03
[89m 12s] Epoch 94 [28736/50000] loss = 0.03
[89m 22s] Epoch 94 [38336/50000] loss = 0.02
[89m 32s] Epoch 94 [47936/50000] loss = 0.03
Accuracy on test set: 71.52% [7152/10000]
[89m 47s] Epoch 95 [9536/50000] loss = 0.02
[89m 58s] Epoch 95 [19136/50000] loss = 0.02
[90m 8s] Epoch 95 [28736/50000] loss = 0.02
[90m 18s] Epoch 95 [38336/50000] loss = 0.02
[90m 29s] Epoch 95 [47936/50000] loss = 0.03
Accuracy on test set: 71.04% [7104/10000]
[90m 46s] Epoch 96 [9536/50000] loss = 0.02
[90m 56s] Epoch 96 [19136/50000] loss = 0.02
[91m 7s] Epoch 96 [28736/50000] loss = 0.03
[91m 17s] Epoch 96 [38336/50000] loss = 0.03
[91m 28s] Epoch 96 [47936/50000] loss = 0.04
Accuracy on test set: 71.07% [7107/10000]
[91m 45s] Epoch 97 [9536/50000] loss = 0.05
[91m 57s] Epoch 97 [19136/50000] loss = 0.04
[92m 7s] Epoch 97 [28736/50000] loss = 0.04
[92m 18s] Epoch 97 [38336/50000] loss = 0.03
[92m 28s] Epoch 97 [47936/50000] loss = 0.03
Accuracy on test set: 71.52% [7152/10000]
[92m 44s] Epoch 98 [9536/50000] loss = 0.02
[92m 54s] Epoch 98 [19136/50000] loss = 0.02
[93m 5s] Epoch 98 [28736/50000] loss = 0.02
[93m 16s] Epoch 98 [38336/50000] loss = 0.03
[93m 26s] Epoch 98 [47936/50000] loss = 0.03
Accuracy on test set: 71.33% [7133/10000]
[93m 42s] Epoch 99 [9536/50000] loss = 0.02
[93m 52s] Epoch 99 [19136/50000] loss = 0.02
[94m 3s] Epoch 99 [28736/50000] loss = 0.02
[94m 13s] Epoch 99 [38336/50000] loss = 0.02
[94m 23s] Epoch 99 [47936/50000] loss = 0.03
Accuracy on test set: 71.15% [7115/10000]
[94m 39s] Epoch 100 [9536/50000] loss = 0.04
[94m 50s] Epoch 100 [19136/50000] loss = 0.03
[95m 0s] Epoch 100 [28736/50000] loss = 0.04
[95m 10s] Epoch 100 [38336/50000] loss = 0.03
[95m 21s] Epoch 100 [47936/50000] loss = 0.05
Accuracy on test set: 71.47% [7147/10000]
可以看出：其准确率比LeNet5高出不少，这充分说明了ResNet18的优越性。
二、参考文献
[1] K. He, X. Zhang, S. Ren and J. Sun. Deep Residual Learning for Image Recognition[C]. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770-778.