A brief introduction to Normalization
This article will briefly introduce the following four Normalizations:
- Batch Norm
- Layer Norm
- Instance Norm
- Group Norm
The specific diagram is as follows ( the diagram is from Brother Tongji Zihao at Station B ):
- Batch Norm: Standardize a certain channel of N pictures (ie 1 batch) .
- Layer Norm: Normalize all channels of an image .
- Instance Norm: Standardize a channel of a picture .
- Group Norm: Between Instance Norm and Layer Norm , standardize certain channels of a picture .
You're done! Finally, thank you for your study~
