Why can the residual network ResNet train a 1000-layer model to learn deep learning v2

1. Residual network ResNet

Assuming that you only know one network in the CNN convolutional neural network, that is the residual network ResNet. ResNet is a very simple and useful network.

Does adding more layers always improve accuracy? It is not the same. For example, F6 on the left is farther from the objective function f* than F3, so the effect is worse.

residual blockf(x) = x + g(x)

https://cv.gluon.ai/model_zoo/classification.html

2. Code implementation

10个epoch，train acc 0.996， test acc 0.885

3. Why can ResNet train a 1000-layer model

Because multiplication will make the gradient disappear or the gradient will explode; assuming that the gradient can be controlled to disappear, ResNet turns the cumulative multiplication into an addition, which solves the problem of gradient disappearance.

4. Q&A

• 1. Residual network ResNet f(x) = x + g(x), if g(x)the network training is changed, doesn't it weaken this effect? Yes, the residual network ResNet is to ensure that it will not deteriorate under the deep neural network.
• 2. Why can't ImageNet achieve 100% test accuracy. Because of the test set in ImageNet, some labels are wrong.

5. The second part ends the competition: image classification

Competition address: https://www.kaggle.com/c/classify-leaves

reference

https://www.bilibili.com/video/BV1bV41177ap?p=1