Disclaimer: This article is a blogger original article, follow the CC 4.0 BY-SA copyright agreement, reproduced, please attach the original source link and this statement.
Scale Layer is input to zoom and pan, often appear in the normalized BatchNorm, Caffe BatchNorm + Scale commonly used in the normalization operation (equivalent Pytorch in BatchNorm)
First we look at ScaleParameter
message ScaleParameter {
// The first axis of bottom[0] (the first input Blob) along which to apply
// bottom[1] (the second input Blob). May be negative to index from the end
// (e.g., -1 for the last axis).
// 根据 bottom[0] 指定 bottom[1] 的形状
// For example, if bottom[0] is 4D with shape 100x3x40x60, the output
// top[0] will have the same shape, and bottom[1] may have any of the
// following shapes (for the given value of axis):
// (axis == 0 == -4) 100; 100x3; 100x3x40; 100x3x40x60
// (axis == 1 == -3) 3; 3x40; 3x40x60
// (axis == 2 == -2) 40; 40x60
// (axis == 3 == -1) 60
// Furthermore, bottom[1] may have the empty shape (regardless of the value of
// "axis") -- a scalar multiplier.
// 例如，如果 bottom[0] 的 shape 为 100x3x40x60，则 top[0] 输出相同的 shape；
// bottom[1] 可以包含上面 shapes 中的任一种(对于给定 axis 值).
// 而且，bottom[1] 可以是 empty shape 的，没有任何的 axis 值，只是一个标量的乘子.
optional int32 axis = 1 [default = 1];
// (num_axes is ignored unless just one bottom is given and the scale is
// a learned parameter of the layer. Otherwise, num_axes is determined by the
// number of axes by the second bottom.)
// (忽略 num_axes 参数，除非只给定一个 bottom 及 scale 是网络层的一个学习到的参数.
// 否则，num_axes 是由第二个 bottom 的数量来决定的.)
// The number of axes of the input (bottom[0]) covered by the scale
// parameter, or -1 to cover all axes of bottom[0] starting from `axis`.
// Set num_axes := 0, to multiply with a zero-axis Blob: a scalar.
// bottom[0] 的 num_axes 是由 scale 参数覆盖的；
optional int32 num_axes = 2 [default = 1];
// (filler is ignored unless just one bottom is given and the scale is
// a learned parameter of the layer.)
// (忽略 filler 参数，除非只给定一个 bottom 及 scale 是网络层的一个学习到的参数.
// The initialization for the learned scale parameter.
// scale 参数学习的初始化
// Default is the unit (1) initialization, resulting in the ScaleLayer
// initially performing the identity operation.
// 默认是单位初始化，使 Scale 层初始进行单位操作.
optional FillerParameter filler = 3;
// Whether to also learn a bias (equivalent to a ScaleLayer+BiasLayer, but
// may be more efficient). Initialized with bias_filler (defaults to 0).
// 是否学习 bias，等价于 ScaleLayer+BiasLayer，只不过效率更高
// 采用 bias_filler 进行初始化. 默认为 0.
optional bool bias_term = 4 [default = false];
optional FillerParameter bias_filler = 5;
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
Scale layer inside the prototxt writing:
layer {
name: "scale_conv1"
type: "Scale"
bottom: "conv1"
top: "conv1"
scale_param {
bias_term: true
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
For example, in MobileNet in:
layer {
name: "conv6_4/scale"
type: "Scale"
bottom: "conv6_4/bn"
top: "conv6_4/bn"
param {
lr_mult: 1
decay_mult: 0
}
param {
lr_mult: 1
decay_mult: 0
}
scale_param {
bias_term: true
}
}
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17