mobilenet V3论文笔记

论文亮点：

1、优化激活函数（可用于其他网络结构）

We settled on using the hard swish nonlinearity for this layer
as it performed as well as other nonlinearities tested. We
were able to reduce the number of filters to 16 while maintaining the same accuracy as 32 filters using either ReLU
or swish. This saves an additional 3 milliseconds and 10
million MAdds.
 

深层次的卷积层后可以用这个激活函数替代传统的激活函数，如RELU等。减小卷积的通道数的情况下采用这个激活函数可以保持减少通道数之前的精度，与此同时又减少了计算量。

详情见：

【1】P. Ramachandran, B. Zoph, and Q. V. Le. Searching for activation functions. CoRR, abs/1710.05941, 2017. 2, 4