Tensorflow2.0 in tf.keras.layers.Conv2D in the initialization method 'glorot_uniform' in the end is Gesha?

We use tf.keras.layers.Conv2D time to build layer convolution weights initialization method generally used method is the default value of this function, i.e. 'glorot_uniform' .
Its source code to make an explanation:

'''
  It draws samples from a uniform distribution within [-limit, limit]
  where `limit` is `sqrt(6 / (fan_in + fan_out))`
  where `fan_in` is the number of input units in the weight tensor
  and `fan_out` is the number of output units in the weight tensor.
'''

That is, uniform distribution of this range initialization method using [-limit, limit] , wherein,
l i m i t = s q r t ( 6 f a n i n + f a n o u t ) limit=sqrt(\frac{6}{fan_{in} + fan_{out}})
Here f a n i n fan_{in} It represents the number of input neurons, f a n o u t fan_{out} It indicates the number of output neurons, namely:
f a n i n = c h a n n e l s i n × k e r n e r w i d t h × k e r n e r h e i g h t fan_{in}​=channels_{in}​\times kerner_{width}\times kerner_{height}​​
f a n o u t = c h a n n e l s o u t × k e r n e r w i d t h × k e r n e r h e i g h t fan_{out​}=channels_{out​}\times kerner_{width}​\times kerner_{height}​
Assume that the input of a network28 * 28 * 1data, the shape of the convolution kernel is a3 * 3convolution core has channels32th (i.e., the number of output channels with a32th), then the timelimitis:
l i m i t = s q r t ( 6 3 × 3 × 1 + 3 × 3 × 32 ) limit=sqrt(\frac{6}{3\times 3\times 1 + 3\times 3\times 32})

Published 136 original articles · won praise 34 · Views 100,000 +

Guess you like

Origin blog.csdn.net/qq_36758914/article/details/105315547