Most neural network training process is relatively long, in order to avoid accidents during training results lead to the loss, we need to adopt methods to save the training process. In addition, the items were not predict how many times the training to achieve the proper accuracy, so often have to first save the time of training results at the end of a program of training, and then to decide whether to continue training based on current error rate, then again when training We need to save the results of the training load retraining down.
Of course, if the result of poor training, but also the network structure may be adjusted to start training again, this section will introduce methods to save and load training process.
First, save the training process
The following code to issue identity cards as a case, save the training process.
The following execution of the program named: idcard4.py
Import tensorflow TF AS Import Random random.seed () # define input variables x layer tensor formula x = tf.placeholder (DTYPE = tf.float32) # define the output layer scalar tensor yYrain formula yTrain = tf.placeholder (dtype = tf.float32) # define w1: a first layer of variable weighting parameter, the input layer with four nodes, there are eight nodes in the hidden layer, it is w1 [4,8] matrix, convincing normal w1 = tf.Variable (tf.random_normal ([4,8], Mean = 0.5, STDDEV = 0.1), DTYPE = tf.float32) # define b1 offset, phase addition and subtraction operations and the vector or matrix, each will a vector or matrix of each element in the same operation as B1 tf.Variable = (0, DTYPE = tf.float32) # define xr: converting the input data x is a vector from a thinking [1,4] of the matrix , xr stored in the variable, the matrix multiplication operation performed later. tf.reshape = XR (X, [l, 4 ]) #n1: 1 hidden layer structure, tf.matmul function matrix multiplication, tf.nn.tanh activation function is tanh, to the linear effect. tf.nn.tanh = N1 (tf.matmul (XR, W1) + B1) # define w2: weight of the second layer of the variable value of the parameter, a first hidden layer nodes 8, the second hidden layer 2 node, then w1 is [8,2] matrix, normal convincing # define the second hidden layer 2, it is to allow the output to the output layer nodes is 2, to facilitate application sorftmax function w2 = tf. Variable (tf.random_normal ([8,2], Mean = 0.5, STDDEV = 0.1), DTYPE = tf.float32) # define offset b2 b2 tf.Variable = (0, DTYPE = tf.float32) # hidden layer structure 2 N2 = tf.matmul (N1, w2 of) + B2 Y = tf.nn.softmax (tf.reshape (N2, [2 ])) # use the averaging function evaluation error squared error loss = tf.reduce_mean (tf .square (Y- yTrain)) Optimizer = tf.train.RMSPropOptimizer (0.01 ) Train = optimizer.minimize (Loss) # define a variable sess, call the session object sess = tf.Session () # define a variable init, his return is the object of a dedicated variable initialization parameter, init = tf.global_variables_initializer () # call session object sess run function of run () sess.run (the init) lossSum = 0.0 for I in Range (. 5 ): xDataRandom = [int (random.random () * 10), int (random.random ( ) * 10), int (random.random () * 10), int (random.random () * 10 )] IF xDataRandom [2]% 2 == 0: yTrainDataRandom = [0,1 ] the else : yTrainDataRandom = [1,0] result = sess.run([train,x,yTrain,y,loss],feed_dict={x:xDataRandom,yTrain:yTrainDataRandom}) #print(result) lossSum = lossSum + float(result[len(result)-1]) print('i:%d,loss:%10.10f,avgLoss:%10.10f'%(i,float(result[len(result)-1]),lossSum/(i+1)))
trainResultPath = './save/idcard2' print('saving...') tf.train.Saver().save(sess,save_path=trainResultPath)
Slowly reading the code, you can find the training process is to save the code:
trainResultPath = './save/idcard2' print('saving...') tf.train.Saver().save(sess,save_path=trainResultPath)
Used to specify a scalar trainResultPath stored data records in the training process, '/ save / idcard2.': Save subdirectory location is stored a program executed when the next directory in idcard4.py idcard2 text base name.
Call the member function Saver save the object under tensorflow train Saver function returns the next package to be saved, the first argument of the current session object, the second parameter is passed to save path location.
Second, load a saved training process and continue training
The code is: idcard4.py
# Load the saved training process to continue training and Import tensorflow TF AS Import Random Import OS trainResultPath = ' ./save/idcard2 ' random.seed () # define input variables x layer tensor formula x = tf.placeholder (dtype = tf.float32) # define the output layer scalar tensor yYrain formula yTrain = tf.placeholder (DTYPE = tf.float32) # define w1: a first layer of variable weighting parameter, the input layer with four nodes, hidden layer 8 nodes, then w1 is [4,8] matrix, convincing normal w1 = tf.Variable (tf.random_normal ([4,8], Mean = 0.5, STDDEV = 0.1), DTYPE = tf.float32) # define offset b1, with vector or matrix operations such as addition and subtraction, would each vector or matrix of each element in the same operation as b1 = tf.Variable (0, DTYPE = tf.float32) #Xr defined: the conversion from the input data x is a vector of a thinking [1,4] of the matrix, xr stored in a variable, the matrix multiplication operation performed later. tf.reshape = XR (X, [l, 4 ]) # N1: hidden layer structure 1, tf.matmul function matrix multiplication, tf.nn.tanh activation function is tanh, to the linear effect. tf.nn.tanh = N1 (tf.matmul (XR, W1) + B1) # define w2: weight of the second layer of the variable value of the parameter, a first hidden layer nodes 8, the second hidden layer 2 node, then w1 is [8,2] matrix, normal convincing # define the second hidden layer 2, it is to allow the output to the output layer nodes is 2, to facilitate application sorftmax function w2 = tf. Variable (tf.random_normal ([8,2], Mean = 0.5, STDDEV = 0.1), DTYPE = tf.float32) # define offset b2 b2 tf.Variable = (0, DTYPE = tf.float32) # hidden layer structure 2 N2 = tf.matmul (N1, w2 of) + B2 # first aspect of the second hidden layer matrix [1,2] is converted to a vector [2] is converted into a two-dimensional vector Y = tf.nn .softmax (tf.reshape (n2, [2])) # Use the averaging function evaluation error square error Loss = tf.reduce_mean (tf.square (Y- yTrain)) Optimizer = tf.train.RMSPropOptimizer (0.01 ) Train = optimizer.minimize (Loss) # define a variable sess call session object sess = tf.Session () # define a variable init, his return is the object of a dedicated variable initialization parameters, # init = tf.global_variables_initializer () # call the session object sess the run function run ( ) # sess.run (the init) IF os.path.exists (trainResultPath + '. index'): Print ( 'loading:% S' trainResultPath%) . tf.train.Saver () Restore (Sess, the save_path = trainResultPath) the else : Print ( 'Not EXISTS path Train Result: S%'% trainResultPath) sess.run(tf.global_variables_initializer()) lossSum = 0.0 for i in range(5): xDataRandom = [int(random.random()*10),int(random.random()*10),int(random.random()*10),int(random.random()*10)] if xDataRandom[2]%2 ==0: yTrainDataRandom = [0,1] else: yTrainDataRandom = [1,0] result = sess.run([train,x,yTrain,y,loss],feed_dict={x:xDataRandom,yTrain:yTrainDataRandom}) #print(result) lossSum = lossSum + float(result[len(result)-1]) print('i:%d,loss:%10.10f,avgLoss:%10.10f'%(i,float(result[len(result)-1]),lossSum/(i+1))) print('saving......') tf.train.Saver().save(sess,save_path = trainResultPath)
After reading the above code, you can see there are two and save the training process is not the same place:
1, the statement trainResultPath variables defined in the program beginning, because when loading the training data also need to use information data file save location.
2, in the definition of a good sess session variables, do not want the same as before immediately calling sess.run (tf.global_variables_initializer ()) to initialize the variable, because if you want to load the training data, initialization can not do otherwise, all of variable parameter has been restored to an initial value. Plus a conditional, if the directory specified location to save the training process the file, save the file to load in the training process information; otherwise, before going to operate variable initialization parameters.
Third, through the command line parameter controls forced to re-start training
The so-called command-line parameters: cmd window operation in this operation file, given a parameter running in a running program.
To give an example:
A self text.py program:
Import SYS argt = sys.arge Print (Arge) Print ( ' printing are: S% ' % Arge [. 1])
In this path location python program, input python text.py abc
Output is:
['text.py', 'abc']
Parameter 1:abc
So, you can execute the saved file using the command-line training process parameters.
Code named: incard6.py
import tensorflow as tf import random import os import sys ifRestartT = False argt = sys.argv[1:] for v in argt: if v == "-restart": ifRestartT = True trainResultPath = "./save/idcard2" random.seed() x = tf.placeholder(tf.float32) yTrain = tf.placeholder(tf.float32) w1 = tf.Variable(tf.random_normal([4, 8], mean=0.5, stddev=0.1), dtype=tf.float32) b1 = tf.Variable(0, dtype=tf.float32) xr = tf.reshape(x, [1, 4]) n1 = tf.nn.tanh(tf.matmul(xr, w1) + b1) w2 = tf.Variable(tf.random_normal([8, 2], mean=0.5, stddev=0.1), dtype=tf.float32) b2 = tf.Variable(0, dtype=tf.float32) n2 = tf.matmul(n1, w2) + b2 y = tf.nn.softmax(tf.reshape(n2, [2])) loss = tf.reduce_mean(tf.square(y - yTrain)) optimizer = tf.train.RMSPropOptimizer(0.01) train = optimizer.minimize(loss) sess = tf.Session() if ifRestartT == True: print("force restart...") sess.run(tf.global_variables_initializer()) elif os.path.exists(trainResultPath + ".index"): print("loading: %s" % trainResultPath) tf.train.Saver().restore(sess, save_path=trainResultPath) else: print("train result path not exists: %s" % trainResultPath) sess.run(tf.global_variables_initializer()) lossSum = 0.0 for i in range(5): xDataRandom = [int(random.random() * 10), int(random.random() * 10), int(random.random() * 10), int(random.random() * 10)] if xDataRandom[2] % 2 == 0: yTrainDataRandom = [0, 1] else: yTrainDataRandom = [1, 0] result = sess.run([train, x, yTrain, y, loss], feed_dict={x: xDataRandom, yTrain: yTrainDataRandom}) lossSum = lossSum + float(result[len(result) - 1]) print("i: %d, loss: %10.10f, avgLoss: %10.10f" % (i, float(result[len(result) - 1]), lossSum / (i + 1))) print("saving...") tf.train.Saver().save(sess, save_path=trainResultPath)
In cmd window, with the result python incard6.py -restart command line to execute the program, run as follows:
the restart ... Force I: 0, Loss: .2178457528, avgLoss: 0.3178457528 I:. 1, Loss: .1825756580, avgLoss: .2002107054 I: 2, Loss: .3673197329, avgLoss: .2559137146 I:. 3, Loss: 0.1935390979, avgLoss: .2403200604 I : 4, Loss: .1634049714, avgLoss: .2249370426 Saving ......
Fourth, read from the command line parameters need to predict the data
to train the neural network is to allow the neural network has availability, you need to enter data using a neural network to predict the real . These training data as input data and do not want to target, but indeed calculated by the neural network to achieve predicted results.
The following describes the method of data input through the command line parameter prediction target.
import tensorflow as tf import numpy as np import random import os import sys ifRestartT = False predictData = None argt = sys.argv[1:] for v in argt: if v == "-restart": ifRestartT = True if v.startswith("-predict="): tmpStr = v[len("-predict="):] predictData = np.fromstring(tmpStr, dtype=np.float32, sep=",") print("predictData: %s" % predictData) trainResultPath = "./save/idcard2" random.seed() x = tf.placeholder(tf.float32) yTrain = tf.placeholder(tf.float32) w1 = tf.Variable(tf.random_normal([4, 8], mean=0.5, stddev=0.1), dtype=tf.float32) b1 = tf.Variable(0, dtype=tf.float32) xr = tf.reshape(x, [1, 4]) n1 = tf.nn.tanh(tf.matmul(xr, w1) + b1) w2 = tf.Variable(tf.random_normal([8, 2], mean=0.5, stddev=0.1), dtype=tf.float32) b2 = tf.Variable(0, dtype=tf.float32) n2 = tf.matmul(n1, w2) + b2 y = tf.nn.softmax(tf.reshape(n2, [2])) loss = tf.reduce_mean(tf.square(y - yTrain)) optimizer = tf.train.RMSPropOptimizer(0.01) train = optimizer.minimize(loss) sess = tf.Session() if ifRestartT: print("force restart...") sess.run(tf.global_variables_initializer()) elif os.path.exists(trainResultPath + ".index"): print("loading: %s" % trainResultPath) tf.train.Saver().restore(sess, save_path=trainResultPath) else: print("train result path not exists: %s" % trainResultPath) sess.run(tf.global_variables_initializer()) if predictData is not None: result = sess.run([x, y], feed_dict={x: predictData}) print(result[1]) print(y.eval(session=sess, feed_dict={x: predictData})) sys.exit(0) lossSum = 0.0 for i in range(5): xDataRandom = [int(random.random() * 10), int(random.random() * 10), int(random.random() * 10), int(random.random() * 10)] if xDataRandom[2] % 2 == 0: yTrainDataRandom = [0, 1] else: yTrainDataRandom = [1, 0] result = sess.run([train, x, yTrain, y, loss], feed_dict={x: xDataRandom, yTrain: yTrainDataRandom}) lossSum = lossSum + float(result[len(result) - 1]) print("i: %d, loss: %10.10f, avgLoss: %10.10f" % (i, float(result[len(result) - 1]), lossSum / (i + 1))) if os.path.exists("save.txt"): os.remove("save.txt") print("saving...") tf.train.Saver().save(sess, save_path=trainResultPath) resultT = input('Would you like to save? (y/n)') if resultT == "y": print("saving...") tf.train.Saver().save(sess, save_path=trainResultPath)
predictData:[1,2,3,4]
loading:./save.idcard2 [0.32765886 0.67232467]
[0.32765886 0.67232467]