Basics for the neural network

                                   

Getting the basics of neural network neural networks basics

Perhaps now refer deep learning (deep learning) and even non-computer professionals have heard of, especially in the last "man-machine war" is set off great concern to the depth of learning. The depth study of neural networks is derived. Neural network is actually a relatively old method, the purpose it was originally produced by a manufacturer can simulate the planet's most complex thing "brain" of the machine, but also the development of neural networks twists and turns:

The first high tide : the twentieth century MP neural network model forties, after Hebb learning law appears fifties appeared to Perceptron, Adaline, represented by a series of results, can be seen as the first development of neural networks a climax.

Ice Age : Figure prize winner, founder of MIT computer science Marvin MinsKy (Min Marvin Chomsky) (January 24, 2016 died) published "perception machine," a book in 1969 that a single layer neural network can not resolve the non linear problem, and multilayer neural network training algorithm still do not see hope. This statement led directly into the neural network 10 years of ice age.

The second peak period : rapid popularity BP algorithm, setting off a second climax neural network.

Ice Age: the mid-1990s, with the statistical learning theory and SVM (support vector machine, SVM) the rise of the neural network re-enter underestimated, NIPS conference neural network for many years does not accept the theme of the paper.

The third peak period : After 2010, with the advent of the rapid increase in computing power and big data, neural network research in the name of "deep learning" renewal New Year again.

If you find this article also looks a little bit hard, or want to learn the system of artificial intelligence, it is recommended that you go to bed long artificial intelligence tutorial. For the great big God, tutorials not only easy to understand, and very humorous. Click here you can view the tutorial.

Is a nonlinear neural network learning algorithm, the neural network is the most basic component of neurons (Neuron), neuron model gives substantially below:

Wherein   the input unit,  called a bias unit (bias unit),  referred to as connection weights. Output function, which , in fact, mentioned earlier in the logistic regression in the sigmoid function (ps: the current output layer neural network typically use softmax function). 

Remember the sigmoid function logistic regression we do, referred to herein as "excitation function" (motivation function), whose image is (images from wiki):

About this function will not introduce more, which I introduced in the logistic regression in.

After introduction neuron model, to introduce the following basic neural network model:

In the above figure I did not draw bias unit , in practical applications sometimes need to add. Wherein the first layer (Layer1) called the input layer, a hidden layer called Layer2 (neural networks may be more than one hidden layer, but the present embodiment is only one, as long as the fact is called hidden layers located between the hidden layer and output layer input ), layer3 called output layer. J excitation layer is referred to as i-th unit , it represents the j-th layer to the first layer j + 1 heavy weight. Thus in the present example:

Therefore here  . It can be seen   is a 3 * 4 matrix, namely:

If a neural network in the j layer units, the first layer j + 1 units, so as matrix.

The most basic knowledge of neural networks on the introduction, here are some examples to help you better understand the neural network.

Example 1: to achieve "and" function

As shown above, we are easily obtained   . Therefore, we can draw the truth table and see how it is to achieve "and" function.

Thus, it can be seen from the figure is how to achieve "and" function. . .

Example 2:

Achieve "or" function:

注：写博客不容易，甚是花费时间和精力，尤其画图很不容易。每个原创博客的孩子上辈子都是天使，请大家尊重原创，转载或者引用请注明原文作者和链接，谢谢！