When I modified the network by myself, I found that there were both add and cat operations in skip connection.
The task I am doing is similar to image restoration. Anyway, the image is first encoder, then resnet and then decoder. I accidentally found that there is a situation in the output image that every test set image after inference has a small black hole, and each has an explanation. It is not an accident.
Through comparative experiments, it is found that when the other settings are the same and skip connection is add, the model inference output image will basically have a small black hole in the lowest part of the image (close to black). But if you use cat operation, this will not happen.
Later, I realized that I saw the explanation of the difference between add and cat. https://www.zhihu.com/question/306213462 I felt that the explanation was clear and easy to understand. Without strict control of computing resources, I felt it was better to use cat. After all, add The situation is learned with cat, but add thinks that it limits other possibilities.