MNIST data processing
It contains 60,000 data as training data, 10,000 data as the test data. Data set each image represents a number of 0-9, image size are 28 * 28, numbers are right in the middle of the picture. tensorflow provides a class to handle MNIST data.
one-hot encoding
One-Hot encoding, also known as an efficient coding, is used mainly N-bit status registers to encode the N state, and only one valid at any time. One-Hot encoding is represented as binary categorical variables vector. This requires classification values are mapped to integer values. Then, each of the integer value is represented as a binary vector, in addition to integer indexes, which are set to zero, it is marked as 1.
1 from tensorflow.examples.tutorials.mnist import input_data 2 3 mnist = input_data.read_data_sets('path/to/MNIST_data/',one_hot=True) #得到一个mnist类 4 5 print ('Training data size:',mnist.train.num_examples) 6 print('Validation data size:',mnist.validation.num_examples) 7 print('Testing data size:',mnist.test.num_examples) 8 9 a =mnist.train.images [0] 10 # 28 * 28 = 784 = size, pixel matrix is in the range [0,1], representing the color depth, 0 for white background represents a black background . 11 Print ( ' Example Training Data: ' , A, ' \ n- ' , ' size: ' , a.shape) 12 is 13 is B = mnist.train.labels [0] 14 Print ( ' Example Training Data label: ' , ' \ n- ' , B, ' size: ' , b.shape)
On the book moving average, decreased learning rate, regularization have used the model of training, and compared the prediction accuracy.
Here I did not use these techniques, carried out the most simple training, the code is as follows:
. 1 # - * - Coding: UTF-. 8 - * - 2 "" " . 3 the Created 23 is ON Wed On Oct 2019 21:25:30 . 4 . 5 to create a three-layer fully connected network, a digital identification. 6 uses learning rate decay , L2 regularization, moving average model to train the network . 7 "" " . 8 Import tensorflow TF AS . 9 from tensorflow.examples.tutorials.mnist Import Input_Data 10 . 11 INPUT_NODE 784 = 12 is OUTPUT_NODE 10 = 13 is # hidden layer nodes 14 LAYER1_NODE = 500 15 BATCH_SIZE = 100 16 LEARNING_RATE_BASE = 0.8 . 17 LEARNING_RATE_DECAY = 0.99 18 REGULARIZATION_RATE = 0.0001 19 TRAINING_STEPS = 30000 20 MOVING_AVERAGE_DECAY = 0.99 21 learning_rate = 0.001 22 #定义网络的前向传播,avg_class为滑动平均类 23 def feed_forward(input_tensor,avg_class,weights1,bias1,weights2,bias2): 24 if avg_class==None: 25 layer1 = tf.nn.relu(tf.matmul(input_tensor,weights1) + bias1) 26 return tf.matmul(layer1,weights2) + bias2 27 else: 28 = tf.nn.relu Layer1 (tf.matmul (input_tensor, avg_class.average (weights1)) + avg_class.average (Bias1)) 29 return tf.matmul (Layer1, avg_class.average (weights2)) + avg_class.average (Bias2 ) 30 31 is # training model process 32 # DEF train (MNIST): 33 is X = tf.placeholder (tf.float32, [None, INPUT_NODE], name = ' X-INPUT ' ) 34 is Y_ = tf.placeholder (TF. float32, [None, OUTPUT_NODE], name = ' the Y-Output ' ) 35 # generated hidden layer parameters tf.random_normal 36 # is different in that it tf.truncated_normal average of greater than two standard deviation values will be discarded and reselect 37 [ weights1 = tf.Variable (tf.random_normal ([INPUT_NODE, LAYER1_NODE], DTYPE = tf.float32)) 38 is Bias1 = tf.Variable (tf.constant (0.1, Shape = [LAYER1_NODE])) 39 weights2 = tf.Variable (tf.random_normal ([LAYER1_NODE, OUTPUT_NODE], DTYPE = tf.float32)) 40 Bias2 = tf.Variable (tf.constant (0.1, Shape = [OUTPUT_NODE])) 41 is Y = feed_forward (X, None, weights1, Bias1 , weights2, Bias2) 42 is global_step tf.Variable = (0, trainable = False) 43 is 44 is # moving average portion, the moving average based .apply (variable) calculated moving average value, the moving average based .average (variable) moving average call ? 45 # tf.argmax (tensor, axis) axis dimension returns the index of the maximum of the tensor 46 is 47 # calculates L2 regularization loss function, generally calculate the weight of the right side of the neural network regularization loss, not counting the bias term 48 # Labels should be equal to the rank of the rank logits -1 49 cross_entropy tf.nn.sparse_softmax_cross_entropy_with_logits = ( = tf.argmax Labels (Y_,. 1), logits = Y) 50 Loss = cross_entropy 51 is 52 is train_step = tf.train.GradientDescentOptimizer (learning_rate) .minimize (Loss, global_step = global_step) 53 is 54 is # use of a moving average, every require both data again to update the parameters in the neural network through back-propagation, but also to update the moving average of each parameter. 55 # to a plurality of operations at once, tensorflow provided tf.group and tf.contol_dependencies mechanism 56 is 57 is # tf.cast (True, tf.float32) = 1.0 tf.cast (False, tf.float32) = 0.0 58 correct_prediction = tf.equal(tf.argmax(y,1),tf.argmax(y_,1)) 59 accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32)) 60 61 with tf.Session() as sess: 62 tf.initialize_all_variables().run() 63 64 validate_feed = {x:mnist.validation.images,y_:mnist.validation.labels} 65 test_feed = {x:mnist.test.images,y_:mnist.test.labels} 66 67 for i in range(TRAINING_STEPS): 68 if i % 1000 == 0: 69 validation_acc = sess.run(accuracy,feed_dict=validate_feed) 70 Print ( ' After Training% Steps D, G Validation% Accuracy IS ' % (I, validation_acc)) 71 is # MNIST class provides functions next_batch 72 XS, YS = mnist.train.next_batch (BATCH_SIZE) 73 is sess.run (train_step, feed_dict = {X: XS, Y_: YS}) 74 75 # after the training, the test set was tested on 76 test_acc = sess.run (Accuracy, feed_dict = test_feed) 77 Print ( ' after training Steps D% , IS% Accuracy Test G ' % (TRAINING_STEPS, test_acc)) 78 79 #主程序 80 #def main(argv=None): 81 # mnist = input_data.read_data_sets('/mnistdata',one_hot = True) 82 #train(mnist) 83 84 #if __name__ == '__main__': 85 # tf.app.run() 86