Neural Networks net Lesson Study Notes

TensorFlow basic usage

Parameters (weight)

例： w=tf.Variable(tf.random_normal([2,3],stddev=2,mean=0,seed=1)

tf.random_normal () normal distribution
[2,3] generate a matrix of 2 * 3
stddev = 2 standard deviation of 2
mean = 0 mean 0
seed = 1 random seed, if set to 0, then each random number different

tf.truncated_normal () remove excessive deviation of the point from the average standard deviation of more than two points are regenerated
tf.random_uniform () evenly distributed
tf.ones () 1 full array
tf.zeros)) the full array tf 0 .fill () array of full value, such as tf.fill ([2,3], 6) , generates a full 3 * 2 array 6
tf.constant () directly to the value tf.constant ([1,2 3]), generating [1,2,3]

The basic realization of neural networks

Preparing a data set, extracting a feature as an input, the neural network pass
NN building structure, from input to output (forward propagation - calculating an output> build calculation map, then execution Sess)
A number of feature data fed NN, NN iterative optimization parameters (back-propagation -> Optimization parameters training model)
Using the trained model prediction and classification (sometimes use was set for the network, feeding new data, and then optimize the parameters of a)
two processes: the training process and use

Forward propagation (fully connected neural network)

Construction model -> realization reasoning

Suppose x is the input, the matrix is a 1 * 2 (two features) generally refers to layers refers computing layer, the input layer than computational layer.

Initializes the variable (Variable) of the
type in which sess tf.global_variables_initializer ()
for a placeholder tf.placeholder (), temporarily to a value, with which the sess feed_dict = (x: ...)
can be fed into a plurality of data , shape of the first dimension can write none

x=tf.placeholder(tf.float32,shape=f(None,2))

init_op=tf.global_variables_initializer()
sess.run(init_op)

Back Propagation

Objective: training the model parameters, using a gradient of decrease in all parameters to minimize the loss of function of the NN model training data

Loss function (loss) = the predicted value (y) and known answer (Y_ (tag provided in advance)) gap

Mean square error (MSE) function can be expressed as a TensorFlow:

loss=tf.reduce_mean(tf.square(y_-y)

Backpropagation training methods: optimization objective is to reduce the loss

Optimizer: choose one when you can use which can be seen, they have an argument, learning_rate learning rate

Learning rate determines the magnitude of each parameter update, you can start to fill up a small value, such as 0.001

Fallback upper neural network is achieved in the
features -> tag, the training set is not as long as the extracted features, but also to its corresponding tag
prior to the transmission, the calculated Y; then reverse propagation loss value calculated

Neural network training standards are based on data and probability, did not know the man-marking method
Examples:

#coding:utf - 8
import tensorflow as tf
import numpy as np  #python的科学计数模块

BATCH_SIZE=5 #表示一次喂入多少组数据,每次不可过大 seed=23455 #基于seed产生随机数 rng=np.random.RandomState(seed) X=rng.rand(32,2)#随机数返回32*2的矩阵，表示32组 体积、重量 做为输入数据集 Y=[[int(x0+x1<1)] for (x0,x1) in X]#生成训练集对应的标签，此处人为给出是否合格的标准(虚拟的样本和标签) print("X:\n",X)#构建出数据集x print("Y:\n",Y)#构建出数据集对应的标签y #定义神经网络的输入、输出和参数 x=tf.placeholder(tf.float32,shape=(None,2)) #输入的特征，不知道多少组，但是知道有两个特征(第一维表示多少组，第二维表示特征个数) y_=tf.placeholder(tf.float32,shape=(None,1)) #标准答案（合格或不合格） #使用正态分布生成随机数 w1=tf.Variable(tf.random_normal([2,3],stddev=1,seed=1)) #输入是两个特征，隐藏层用三个神经元,对应x w2=tf.Variable(tf.random_normal([3,2],stddev=1,seed=1)) #输出是一个数，对应y #前向传播，用矩阵乘法实现 a=tf.matmul(x,w1) y=tf.matmul(a,w2) #反向传播 learning_rate=0.001 loss=tf.reduce_mean(tf.square(y-y_)) train_step=tf.train.GradientDescentOptimizer(learning_rate).minimize(loss)#梯度下降 #生成会话，训练steps轮 with tf.Session() as sess: init_op=tf.global_variables_initializer() sess.run(init_op) #输出训练之前： print("w1:\n",sess.run(w1)) print("w2:\n",sess.run(w2)) steps=3000 for i in range(steps): start=(i*BATCH_SIZE)%32 end=start+BATCH_SIZE sess.run(train_step,feed_dict={x:X[start:end],y_:Y[start:end]})#每轮从X和Y中抽取Start-End个特征和标签 if i%500 == 0:#每500轮打印一次loss值 total_loss=sess.run(loss,feed_dict={x: X,y_:Y}) print("After training %d steps,loss on all data is %g" %(i,total_loss)) print("\nw1:",sess.run(w1)) print("\nw2:",sess.run(w2))

operation result:

X: [[0.83494319 0.11482951]
[0.66899751 0.46594987]
[0.60181666 0.58838408]
[0.31836656 0.20502072]
[0.87043944 0.02679395]
[0.41539811 0.43938369]
[0.68635684 0.24833404]
[0.97315228 0.68541849]
[0.03081617 0.89479913]
[0.24665715 0.28584862]
[0.31375667 0.47718349]
[0.56689254 0.77079148]
[0.7321604 0.35828963]
[0.15724842 0.94294584]
[0.34933722 0.84634483]
[0.50304053 0.81299619]
[0.23869886 0.9895604 ]
[0.4636501 0.32531094]
[0.36510487 0.97365522]
[0.73350238 0.83833013]
[0.61810158 0.12580353]
[0.59274817 0.18779828]
[0.87150299 0.34679501]
[0.25883219 0.50002932]
[0.75690948 0.83429824]
[0.29316649 0.05646578]
[0.10409134 0.88235166]
[0.06727785 0.57784761]
[0.38492705 0.48384792]
[0.69234428 0.19687348]
[0.42783492 0.73416985]
[0.09696069 0.04883936]]
Y: [[1], [0], [0], [1], [1], [1], [1], [0], [1], [1], [1], [0], [0], [0], [0], [0], [0], [1], [0], [0], [1], [1], [0], [1], [0], [1], [1], [1], [1], [1], [0], [1]]
w1: [[-0.8113182 1.4845988 0.06532937]
[-2.4427042 0.0992484 0.5912243 ]]
w2:
[[-0.8113182 1.4845988 ]
[ 0.06532937 -2.4427042 ]
[ 0.0992484 0.5912243 ]]
After training 0 steps,loss on all data is 12.0156
After training 500 steps,loss on all data is 0.696271
After training 1000 steps,loss on all data is 0.53858
After training 1500 steps,loss on all data is 0.483959
After training 2000 steps,loss on all data is 0.454848
After training 2500 steps,loss on all data is 0.438255
w1: [[-0.30248263 0.06204198 0.6001488 ] [-2.0333931 -0.48535204 0.7457628 ]] w2: [[-0.30288324 0.6126571 ] [ 0.13707067 -1.9931065 ] [-0.01895767 0.9599734 ]]
[Finished in 3.0s]

Follow-up can change batch_size and optimizer, which try to better effect

Summary:

Neural network structures: preparation, prequel, back propagation, iterative

Preparation: import module defines constants, change the optimization dataset generated
Forward propagation: define input parameters, output

Input: x = y_ =
Parameters: w1 = w2 =
Output: a = y =

3. Backpropagation: loss function defined, back propagation method

loss=
train_step=

4. Output: Training steps wheel

for i in range():
    start=
    end=
    sess.run(train_step,feed_dict)

常用输出，每隔一定轮数输出一次loss