BN层和CONV层合并

卷积层中

卷积权重: W,卷积偏置:B

卷积层运算:W \times X+B

BN 层中
均值:\mu ,方差:\delta,缩放因子:\gamma,偏移:\beta, 一个较小数(防止分母为0):\epsilon

 \large \mu \leftarrow \tfrac{1}{m}\sum_{i=1}^{m}x_i           \large \sigma^2 \leftarrow \tfrac{1}{m}\sum_{i=1}^{m}(x_i-\mu)^2

\large \hat{x_i} \leftarrow \frac{x_i-\mu}{\sqrt{\sigma^2+\epsilon }}        \large y_i \leftarrow \gamma \hat{x_i} + \beta

BN层和卷积层合并后:

\large \alpha = \frac{\gamma }{\sqrt{\sigma^2+\epsilon }}

\large W_{merged} = W\times \alpha

\large B_{merged} =B\times \alpha+(\beta-\mu\times a)

猜你喜欢

转载自blog.csdn.net/B1009/article/details/82756721
今日推荐