论文地址： Deep Sparse Rectifier Neural Networks
ReLU的好处
- 增加了网络的稀疏性，而稀疏性被证明对网络的表达有好处。
如何善用ReLU
- rectifying activation allows deep networks to achieve their best performance without unsupervised pre-training.
- Still, rectifier networks can benefit from unsupervised pre-training in the context of semi-supervised learning where large amounts of unlabeled data are provided.