Basic Features of Multilayer Perceptrons
In Chapter 1, we learned about the Rosenblatt perceptron, which is essentially a single-layer neural network. This chapter demonstrates that this network is limited to classification problems with linearly separable patterns. Then, in Chapter 3, we study adaptive filtering, using Widrow and Hoff's LMS algorithm. This algorithm is also based on a single linear neuron with adjustable weights, which also limits the computational power of this algorithm. To overcome the practical limitations of perceptrons and LMS algorithms, we consider the well-known neural network architecture of multilayer perceptrons.The following three points reveal the basic characteristics of multilayer perceptrons:
1. Each neuron model in the network contains a differentiable nonlinear activation function.
2. The network includes one or more layers hidden between the input and output neural nodes.
3. The network exhibits a high degree of connectivity, the strength of which is determined by the synaptic weights of the network.
However, these same characteristics also lead to the lack of knowledge about network behavior at this stage. First, the theoretical analysis of multilayer perceptrons is difficult due to the existence of nonlinear distribution and the high connectivity of the network. Second, the use of hidden layers makes the learning process harder. This implies that the learning process must decide which features of the input pattern should be represented by the hidden layer neurons. The learning process is thus made more difficult, as it has to search through a much larger space of possible functions, while having to choose among different representations of the input pattern.
References
(Canada) Simon Haykin (Haijin), translated by Shen Furao, Xu Ye, Zheng Jun, Chao Jing. Neural Networks and Machine Learning [M]. Beijing: Machinery Industry Press, 2011. 1-572