极限学习机(Extreme Learning Machine, ELM)原理详解和MATLAB实现

引言

极限学习机不是一个新的东西，只是在算法（方法）上有新的内容。在神经网络结构上，就是一个前向传播的神经网络，和之前几篇博文讲的意义。

为什么我们需要ELM？

The learning speed of feedforward neural networks is in general far slower than required and it has been a major bottleneck in their applications for past decades. Two key reasons behind may be:

1) the slow gradient-based learning algorithms are extensively used to train neural networks.

2) all the parameters of the networks are tuned iteratively by using such learning algorithms.

最大的创新点：

1）输入层和隐含层的连接权值、隐含层的阈值可以随机设定，且设定完后不用再调整。这和BP神经网络不一样，BP需要不断反向去调整权值和阈值。因此这里就能减少一半的运算量了。

2）隐含层和输出层之间的连接权值β不需要迭代调整，而是通过解方程组方式一次性确定。

研究表明，通过这样的规则，模型的泛化性能很好，速度提高了不少。

一言概之，ELM最大的特点就是对于传统的神经网络，尤其是单隐层前馈神经网络(SLFNs)，在保证学习精度的前提下比传统的学习算法速度更快。

Compared BP Algorithm and SVM，ELM has several salient features:

•Ease of use. No parameters need to be manually tuned except predefined network architecture.只有隐含层神经元个数需要我们调整。

•Faster learning speed. Most training can be completed in milliseconds, seconds, and minutes.

•Higher generalization performance. It could obtain better generalization performance than BP in most cases, and reach generalization performance similar to or better than SVM.（泛化能力提升）

•Suitable for almost all nonlinear activation functions.Almost all piecewise continuous (including discontinuous, differential, non-differential functions) can be used as activation functions.

•Suitable for fully complex activation functions. Fully complex functions can also be used as activation functions in ELM.