1. What is a residual connection?
A residual connection is a type of skip connection that adds an input to an intermediate layer or output of the network.
2. What is the role of the residual connection?
The role of the residual connection is to solve the problem of gradient disappearance in the neural network, which can make it easier for the gradient to flow through a deep neural network.
3. What is a residual block?
A residual block is a neural network module composed of residual connections, usually consisting of two or three sets of convolutional layers plus activation functions, and adding residual connections.
4. Why is it said that the residual connection can alleviate the problem of gradient disappearance?
Because the residual connection allows the gradient to be passed directly to the deep layer without passing through the activation function of each layer, it can avoid attenuation in the activation function of each layer too much.
5. What is ResNet?
ResNet is a residual network-based architecture, which achieves high accuracy by adding residual connections to a deep network. ResNet has won the ImageNet competition.
6. What is the typical network structure of ResNet? A
typical ResNet contains multiple residual blocks, and each residual block consists of several sets of convolutional layers plus activation functions. The entire ResNet uses global average pooling and fully connected layers for classification.
7. What kind of connection method is usually used for the residual block?
There are usually two connection methods for the residual block:
1) identity mapping: y = x + F(x)
2) projection mapping: y = x + F(x) * W, where W is a learnable 1x1 convolution
8. What is the naming rule of ResNet?
ResNet uses numbers to indicate the network depth, for example, ResNet-18 means it contains 18 residual blocks, ResNet-50 means it contains 50 residual blocks, and so on.
9. What are the variants of ResNet?
Common ResNet variants include ResNet-D, ResNeXt, SE-ResNet, Res2Net, etc. They improve upon residual connections and residual blocks.
10. What are the application scenarios of ResNet?
ResNet is often used in computer vision tasks such as image classification, image segmentation, and object detection, and has achieved high results in these fields. It is also applied to the task of Natural Language Processing.
11. What is Res2Net? What is the difference between it and ResNet?
Res2Net is an improved ResNet that uses a multi-branch residual structure instead of a single residual path like ResNet. Res2Net can enhance the expressive ability of features and obtain higher accuracy.
12. What is ResNeXt? What is the difference between it and ResNet?
ResNeXt is an improved ResNet that uses group convolution within the residual block to enhance the expressiveness of the model. ResNeXt can be seen as an extended ResNet.
13. What is SE-ResNet? What is the difference between it and ResNet?
SE-ResNet is an improved ResNet that uses a channel attention mechanism (Squeeze-and-Excitation) to reweight channel features. SE-ResNet can make better use of the dependencies between channels to improve accuracy.
14. What is the function of 1x1 convolution in ResNet?
1x1 convolution is mainly used to change the number of channels, which can be used for channel compression or channel expansion, thereby controlling the complexity of the model. In the projection mapping of the residual block, a 1x1 convolution is used to match the number of input and output channels.
15. Why not use the pooling layer in ResNet?
The designer of ResNet found that the use of residual connections can make the network very deep, and good performance can be obtained without the need for a pooling layer. So ResNet removes the maximum pooling layer and only uses a convolution with a stride of 2 for downsampling.
16. What are the implementation details of ResNet?
The main implementation details are: 1x1 convolution using batch normalization, ReLU activation function, residual connection, projection map, etc.
17. In addition to computer vision, what other fields is ResNet used in?
ResNet is also used in natural language processing, such as text classification, machine translation and other tasks, and has achieved good results.
18. What are the disadvantages of ResNet?
The main disadvantage of ResNet is the high computational complexity, especially when the network becomes deeper. Moreover, the residual connection allows the gradient to flow through more easily, which may cause some gradient explosion problems.
19. What is Residual Attention Network?
Residual Attention Network is a network that introduces attention mechanism into ResNet. It adds an attention module on the residual connection, which can better model long-distance dependencies.
20. What is a generalized residual network?
A generalized residual network generally refers to a class of network structures that use the concept of residual learning. In addition to the typical ResNet, it includes ResNeXt, Res2Net, SE-ResNet, Residual Attention Network, etc. These networks all use residual connections and have different improvements on this basis, which can be regarded as generalized residual networks.