White --TensorFlow a deep learning Introduction

                                   

I chose tensorFlow as the first neural network framework of my study, recently Tensorflow happens to support the windows, so let me learn to become more convenient.

A, TensorFlow of running processes

TensorFlow run two-step process, namely the structural model and training.

In the construction phase, we need to build a graph (Graph) to describe our model and then launch it in the session. The so-called FIG, a flowchart will be appreciated, is to process input and output data represented

But this time is not going to happen in the actual operation, because TensorFlow is [delayed execution (deferred execution)] model, it must know what you want to calculate your executed, and then began to send a variety of computing tasks to the computer. So you first use TensorFlow function to create a calculated chart in memory, and then start an executive session and use session.run perform the actual training tasks, such as gradient calculation and other operations, in this case, the map can not be changed.

1.1 Basic Concepts

1.1.1 Tensor

Tensor means that tensor, my understanding is that a number of dimensions indefinite matrix, can also be understood as a representation tensorflow in the matrix, there are many ways to generate tensor, followed by a detailed summary then, for example

   
   
import tensorflow as tf
a=tf.zeros(shape=[ 1, 2])
   
   

   
   

    
    
1
   
   

Note: Before session.run, all data are abstract concepts, that is, a this time but said this should be a zero matrix 1 * 2, but no actual assignment, so if this time print (a) , there will be the following:

   
   
print(a)
#==>Tensor("zeros:0", shape=(1, 2), dtype=float32)
   
   

   
   

    
    
1
   
   

Only after starting session, to get the value of a

   
   
sess=tf.Session()
print (sess.run (a))
[[ 0.  0.]]
   
   

   
   

    
    
1
   
   

If you want the system to study artificial intelligence, it is recommended that you go to bed long artificial intelligence tutorial. For the great big God, tutorials not only easy to understand, and very humorous. Click here you can view the tutorial.

1.1.2. Variable
when the training model of the time, with variables to store and update parameters, variables included tensor (Tensor) stored in the memory buffer, they need to be explicitly initialized modeling, model training after they have b It is stored to disk. The values of these variables can be loaded later in the model training and analysis.

If I want to calculate y = ReLU (Wx + b)

then W, b is the parameter I want to train, then these two values can be represented by Variable, Variable initial function has many options, not to mention here, only input into a th tensor are possible

W=tf.Variable(tf.zeros((1,2)))
   
   

   
   

    
    
1
   
   

in case

   
   
v=tf.Variable(tf.zeros(( 1, 2)))
sess=tf.Session()
#print (sess.run (v)) # will complain, because no variable initialization
sess.run(tf.initialize_all_variables()) # Initialize all variables
print (sess.run (in))
   
   

   
   

    
    
1
   
   

Is output [[0. 0.]]

Sometimes, also can be used to initialize the value of another variable to the current variable initialization. Since tf.initialize_all_variables () to initialize all variables in parallel, so in this demand situation need to be careful.

   
   
# Create a variable with a random value.
weights = tf.Variable(tf.random_normal([ 784 , 200 ], stddev = 0.35 ),
                     name= "weights")
# Create another variable with the same value as 'weights'.
w2 = tf.Variable(weights.initialized_value(), name= "w2")
# Create another variable with twice the value of 'weights'
w_twice = tf.Variable(weights.initialized_value() * 0.2, name= "w_twice")
   
   

   
   

    
    
1
   
   

1.1.3.placeholder placeholder
is also an abstract concept, a format for representing, tell the system input and output data: There is a value / vector / matrix, now I did not send you a specific number, but I officially run time will fill, usually from an external input value, for example, in the example x and y, since no specific value, as long as the size can be specified

   
   
x=tf.placeholder(tf.float32,[ 1, 5],name= 'input')
y=tf.placeholder(tf.float32,[ None, 5],name= 'input')
   
   

   
   

    
    
1
   
   

y represents an input [? 5] matrix, None represents the number of batch input when you need to enter a number to five when it is [5,5] of the matrix, tensorflow will automatically batch.

Note: The only placeholder node is designed with the intent to provide a data feed (feeding) process. when the node is declared placeholder is uninitialized, does not contain data, the data supplied to it if promising, the operation when an error occurs Tensorflow


1.1.4.Session session
Session implementer the abstract model, a specific training parameters, predict, and even the actual value of the query variables to be used in the session


1.2 model building
where the first official said the most basic data set single-layer network example MNIST

(1) reads data

   
   
from tensorflow.examples.tutorials.mnist import input_data
import cv2
import numpy as np

#mnist = input_data.read_data_sets("mnist_data/", one_hot=True)
mnist = input_data.read_data_sets( "C:\\Users\\1\\AppData\\Local\\Programs\\Python\Python35\\Lib\\site-packages\\tensorflow\\examples\\tutorials\\mnist", one_hot= True)
import tensorflow as tf
   
   

   
   

    
    
1
   
   

Here I was directly in the data set from mnist official website, because that code official guidance on net I not take

(2) the establishment of an abstract model

   
   
x = tf.placeholder(tf.float32, [ None, 784])
#W = tf.Variable(tf.random_normal([784, 10], stddev=0.1))
W = tf.Variable(tf.zeros([ 784, 10]))
b = tf.Variable(tf.zeros([ 10]))

y = tf.nn.softmax(tf.matmul(x, W) + b)
y_ = tf.placeholder( "float", [ None, 10])
cross_entropy = -tf.reduce_sum(y_ * tf.log(y))
train_step = tf.train.GradientDescentOptimizer( 0.01).minimize(cross_entropy)
init = tf.global_variables_initializer()
   
   

   
   

    
    
1
   
   

Of the input x, label y_ create a placeholder, and the statement of W, b variables, the prediction result obtained by softmax y
defined loss function cross_entropy, training methods (gradient descent) train_step

(3) practical training

   
   
sess = tf.InteractiveSession()
sess.run (ins)
for i in range( 10000):
   batch_xs, batch_ys = mnist.train.next_batch( 100)
   sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})

   if i % 500== 0:
       correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))
       accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
       print(accuracy.eval(feed_dict={x: mnist.test.images, y_: mnist.test.labels}, session=sess))
   
   

   
   

    
    
1
   
   

After the completion of the model set up, as long as we provide the model inputs and outputs, the model will be able to train and test yourself in the middle of derivation, seeking gradient back-propagation, etc., TensorFlow will help you automatically. The final result was 0.92

Second, the layer fully connected neural network 5

1. First, the definition of each layer neural element number
K = 400
L = 100
M = 60
N = 30


2. construction model

   
   
W1=tf.Variable(tf.truncated_normal([ 28 * 28 , K], stddev = 0.1 ))
B1=tf.Variable(tf.zeros([K]))
W2=tf.Variable(tf.truncated_normal([K,L],stddev= 0.1))
B2=tf.Variable(tf.zeros([L]))
W3=tf.Variable(tf.truncated_normal([L,M],stddev= 0.1 ))
B3=tf.Variable(tf.zeros([M]))
W4=tf.Variable(tf.truncated_normal([M,N],stddev= 0.1 ))
B4=tf.Variable(tf.zeros([N]))
W5=tf.Variable(tf.truncated_normal([N, 10 ], stddev = 0.1 ))
B5=tf.Variable(tf.zeros([ 10]))

X=tf.placeholder(tf.float32,[ None, 28, 28, 1])
y_=tf.placeholder(tf.float32,[ None , 10 ]) # Hot-form Vector
X=tf.reshape(X,[ -1, 28* 28])
Y1=tf.nn.relu(tf.matmul(X,W1)+B1)
Y2=tf.nn.relu(tf.matmul(Y1,W2)+B2)
Y3 = tf.nn.relu (tf.matmul (Y2, W3) + B3)
Y4=tf.nn.relu(tf.matmul(Y3,W4)+B4)
pred=tf.nn.softmax(tf.matmul(Y4,W5)+B5)

loss=tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=pred,labels=y_))
train_step=tf.train.GradientDescentOptimizer( 0.03).minimize(loss)
corr=tf.equal(tf.argmax(pred, 1 ), tf.argmax (Y_, 1 )) # find the maximum output as a result of each row
accu=tf.reduce_mean(tf.cast(corr,tf.float32))
init=tf.global_variables_initializer()
   
   

   
   

    
    
1
   
   

3. Training

   
   
sess=tf.InteractiveSession()
sess.run (ins)
for i in range( 10000):
   batch_xs, batch_ys = mnist.train.next_batch( 100)
   sess.run(train_step, feed_dict={X: batch_xs, y_: batch_ys})
   if i % 100 == 0:
       print( "Test set accuracy:% F" % accu.eval (feed_dict = {X-: mnist.test.images, Y_: mnist.test.labels}, the session = Sess))
   
   

   
   

    
    
1
   
   

Resulting test set accuracy rate: 0.884
The results also no good single-softmax effect is not yet know why ????