Original: http://neuralnetworksanddeeplearning.com/chap1.html#the_architecture_of_neural_networks
The next section I will introduce a well classified handwritten digital neural network. In preparation, we explain the naming of the term in different parts of the network will help later expanded. Suppose we have such a network:
As mentioned earlier, the left-most layer of the network is called input layer , this layer is called the inside of the neuron input neurons ; rightmost output layer contains the output neuron (in this example is an output neuron); intermediate layers, of neurons is neither input nor output, so called hidden layer . "Hide" Maybe there are some mysterious sounds - the first time I heard when I thought it must have a very deep sense of philosophy or mathematics. But it's really in addition to "is not input nor output" than no other meaning. The figure above is only one hidden layer of the network, but the network has a lot more hidden layers. 4 example, the following two-layer network with a hidden layer:
Due to some historical reasons, although the multi-layer network of sigmoid neurons (rather than perceived) is composed, it is sometimes referred to as Multilayer Perceptron ( Multilayer Perceptrons ) or MLPs, puzzling. MLP in this book I do not use the term, because I find it confusing, but still want to warn you that this statement is there.
Input layer and output layer are the straightforward design usually (as determined by the needs). For example, suppose we are trying to determine whether a handwritten picture indicates "9", it is natural that we should be encoded image intensity values for each pixel into the input layer neurons, if the image is a 64 * 64 grayscale, then we 4096 (64 * 64) gradations from 0-1 appropriately scaled input neurons, comprising only one output layer neuron, the output value of less than 0.5 indicates "input image is not a 9", a value greater than 0.5 indicates "input The image is a nine. " (Original text did not write anything equal to 0.5 means)
Although the design of neural network input layer and output layer is usually very simple, but hidden layer design is the job of the arts. In particular, the use of some simple rules of thumb to summarize the hidden layer of the design process is impossible. In contrast, neural network researchers have developed a number of design inspiration for the hidden layer, acts to help people get what they want in their networks. For example, help inspire people to weigh and reduce the number of hidden layers network training time, later in this book we will encounter many such design inspiration. (Design inspired design heuristics is what?)
Now, we have been discussing the output of the neural network layer as the next layer of input, such a network is known as a feed-forward neural network. This means that the network does not loop - information is always passed along, did not return, and if we do have the loop, we'll end (the output end of the day and as their own input in place based on the input function of the output of σ, death cycle, not the end), it does not exist, so we do not allow such a loop.
However, there are some other artificial neural network model, which may have a feedback loop, these models also known as recurrent neural networks , the idea of these models are neurons will operate for a limited period of time and then stop. Those activities will stimulate other neurons, stimulated neurons may also start a bit later, is running for a short time. Over time, we get a cascade of neural activity. This network model, the circuit will not cause a problem, because the output of a neuron will only affect its input in after a certain period of time rather than an instant impact.
Recurrent neural network is smaller than the influence of feed-forward neural networks, in part because of recurrent neural network learning algorithm is not strong enough (at least for now). But the recurrent neural network is still very interesting, they feed the previous neural network closer to our brain works, and may have recurrent neural network can solve some important problems feedforward neural networks is difficult to solve. But the scope of this book because it is not, we will focus on the more common feed-forward neural network.