I. Preamble
As already slowly and gradually "climb" from single neurons to neural networks and the common optimization are resolved one by one, and then move forward is practical application problems, so before you start to have practical application "framework" turned out, because the back to do the work we need to focus on the business rather than the network itself, so using the framework can reduce a lot of work in front have their own experience implementing neural networks, now set up a framework for understanding a number of relatively easy. This article, we will use the more common Tensorflow to reset it in front of the work.
Remarks about Tensorflow installation:
1) install python3.6, high version does not support
2) pip install tensorflow to
Two, softmax
Before you begin you need to say here at the use of a new technology, "softmax" in front of us to solve the problem is to "identify from a bunch of digital pictures in whether it is 9", as used herein softmax we can get a little more difficult issue, such as:
"Identifies a digital picture of a few."
This powerful, and in front of us is not only recognizes that "two-class" here means softmax we can identify a few of the digital picture is the probability that the "multi-classification."
In fact, little changed Technically, the overall neural network structure unchanged, but remember the "activation function" not before we use a neural network? General last layer using sigmoid, which means the output into the interval value between 0 and 1, expressed as "the number 9 is" the probability is. Alternatively Sigmoid softmax used herein, in addition output is not a number, but the number 10, for example:
[0.1, 0.2, 0.3, 0.7, 0.3, 0.1, 0.2, 0.3, 0.1, 0.2]
Its meaning is as follows:
0 probability: 10%
1 probability: 20%
Probability is: 30%
4 of probability: 70%
...
The corresponding label are also dozens of natural input, such as:
[0, 0, 0, 0, 1, 0, 0, 0, 0, 0]
It means that we enter the picture is the number "4", this representation is called "one_hot_label". Such input and output is in the form of "multi-classification" basis, in addition mnist data set we use the label data directly into one hot form, requires only "one_hot_label = True".
In addition to the input and output forms are not the same, softmax spread function and reverse the spread of the sigmoid is certainly not the same, but these powerful means of Tensorflow we do not need to worry about it. Here we have gradually come to realize a neural network-based Tensorflow.
Third, create a placeholder placeholder
In fact, here's input x, y are represented by a vector size of the input image vector label size 784 and 10. tf is tensorflow entities, defined here tf.placeholder is actually an array of two empty:
(784, None)和(10, None)
placeholder to get behind the one-dimensional vector in the role of a "placeholder" placeholder mean building a neural network in the first tensorflow accounted for a good location, just push the placeholder way data entered, threw a real run, such as X is 784, img input no matter what shape you are, anyway, to press 784 to cut into a section of the input.
Fourth, the initialization parameters w, b
On the whole the previous initialization parameters almost change except tf to generate random numbers, and will wGroup bGroup merged Parameters (tf frame only to input a parameter name).
Fifth, the spread function
Here tf.matmul () w is implemented with the IN product matrix, and then () plus b achieved by tf.add. But here the "propagation function" does not really do the spread of computing, it is only by the structure of the neural network of the various operations "arrangements" Well, the last layer operation did not use the activation function to calculate the result, but directly returns A. The remaining operation into the "loss of function" in.
Sixth, the loss of function
tf.transpose here () is just a transpose operation, why not AT this way, in fact, it is conceivable, A here is not a matrix, it is the result of a long list of calculated only when the neural network up and running a will is a matrix. So where A is actually a series of "operator" of the collection, use tf.transpose () is superimposed on a transpose operation.
tf.reduce_mean () Here is the calculation of "loss", but not really being calculated, but this operation "arrangement" Well, the end result is returned as costFun
Seven complete implementation
Etc., but also did not say "back-propagation" mean? Do not panic, here slowly.
In the model, placeholder, initialize_parameters, forward, costCAL are discussed before, only "build" the neural network to process.
optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(costFun)
This sentence is to build back-propagation, which shows that the use of Adam AdamOptimizer loss function optimization algorithm, minimize the use specified, in fact sum up what we need to reverse the spread of Adam optimization algorithm to build the loss function costFun tends to a minimum.
_ , cost = sess.run([optimizer, costFun], feed_dict={X: train_img, Y: train_label})
This is a really a network operation, feed_dict is as previously "placeholder" shape and the train_img train_label input into the network, [optimizer, costFun] is specified network "forward propagation + back propagation" and loss calculations .
parameters = sess.run(parameters)
Only the parameters is output as the optimized back.
Eight, summary
This section will be achieved simply through the use of neural networks before Tensorflow once again achieved, and secondly also introduced softmax will be extended to more than two-class classification. Tensorflow is the basis for subsequent developments, possible to open a separate chapter to talk about.
This section provides a complete implementation of the code number please pay attention to the public "zero-based love learning" Reply AI14 get.
