tensorflow study notes 2 (real)

1、

tf.multiply (x, y1) # multiplied by the corresponding element 
 tf.matmul (x, y2) # matrix multiplication

2, the session: the node performs a calculation operation of FIG.

with tf.Session() as sess:

print sess.run(y)

3 parameters: the weight is w, represented by a variable. Randomized to initial value.

w=tf.Variable(tf.random_normal([2,3],stddev=2,mean=0,seed=1))

The normal distribution, the standard deviation of the mean is 2 0

tf.truncated_normal () remove too large deviation from a normal distribution point

4, the forward spread: to build a model to achieve reasoning

Input layer, hidden layer and output layer

5, variable initialization, calculation of FIG node operation, to be implemented with the session:

Variable initialization:

init_op=tf.global_variables_initializer()

sess.run (init_op)

FIG computing node operation: the data fed by sess.run function feed_dict

6, with tf.placeholder placeholder, the data fed by sess.run function feed_dict

Feeding a set of data:

x = tf.placeholder (tf.float32, shape = (1,2)), then sets of data, the change None 1

sess.run(y,feed_dict={x:[[0.5,0.6]]})

import tensorflow as tf

x = tf.placeholder(tf.float32,shape=(None,2))
w1 = tf.Variable(tf.random_normal([2, 3], stddev=1, seed=1))
w2 = tf.Variable(tf.random_normal([3, 1], stddev=1, seed=1))

a = tf.matmul(x, w1)
y = tf.matmul(a, w2)

with tf.Session() as sess:
    init_op = tf.global_variables_initializer()
    sess.run(init_op)
    print(sess.run(y,feed_dict={x: [[0.7, 0.5],[0.2,0.3],
                                    [0.3,0.4],[0.4,0.5]]}))
    print(sess.run (w1))
     print (sess.run (w2))

7, back propagation: training the model parameters, using a gradient of decrease in all parameters of the NN model on training data

The minimum loss function.

Loss function (loss): predictive value of y and the known gap y_ answer

The mean square error (MSE)

loss=tf.reduce_mean(tf.square(y_-y))

Backpropagation training methods: optimization objective is to reduce the loss

Learning Rate: determine the magnitude of the parameters of each update

Import tensorflow TF AS
 Import numpy AS NP 

BATCH_SIZE =. 8   # primary data fed 
SEED = 23455 

RNG = np.random.RandomState (SEED) 
X- = rng.rand (32,2 ) 
the Y = [[int (X0 + X1 < . 1)] for (X0, X1) in X-]
 Print (X-)
 Print (the Y) 

X = tf.placeholder (tf.float32, Shape = (None, 2)) # size and weight features two 
yy = tf.placeholder (tf.float32, Shape = (None,. 1)) # only one feature, pass or fail 

W1 = tf.Variable (tf.random_normal ([2,3], STDDEV. 1 =, = SEED. 1 )) 
w2 oftf.Variable = (tf.random_normal ([3,1], STDDEV. 1 =, = SEED. 1 )) 

A = tf.matmul (X, W1) 
Y = tf.matmul (A, w2 of) 

# define loss function and inverse the method of propagation 
Loss = tf.reduce_mean (tf.square (yy- Y)) 
train_step = tf.train.GradientDescentOptimizer (from 0.001 ) .minimize (Loss) # learning rate from 0.001 

with tf.Session () AS Sess: 
    init_op = TF .global_variables_initializer () 
    sess.run (init_op) 
    Print (sess.run (W1))
     Print (sess.run (W2)) 

    # training model 
    the STEPS = 3000 # training three thousand 
    for i in the Range (the STEPS): 
        Start= (i*BATCH_SIZE)%32
        end = start+BATCH_SIZE
        sess.run(train_step, feed_dict={x: X[start:end],yy: Y[start: end]})
        if i % 500 == 0:  #每500轮打印一次loss值
           total_loss = sess.run(loss,feed_dict={x: X, yy: Y})
           print(i,total_loss)
        print(sess.run(w1))
        print(sess.run(w2))

8, build neural networks stereotyped: preparation, prequel, back propagation, iterative

(1) Preparation: import; constants defined; generating a data set

(2) Forward propagation: define input and output parameters

(3) back propagation: the definition of a loss function, back propagation method

loss = train_step =

(4) generating a session, training wheel STEPS

9, the loss of function

Complexity NN: NN layers and the number of multi-parameter representation NN

= The number of layers of layers of hidden layer output layer +1

Parameter Total Total = Total w + b

Custom loss function:

Cross entropy: Characterization of the distance between two probability distributions

10, learning rate: the magnitude of each parameter update

How much learning rate settings appropriate? Exponential decay rate

11, the moving average (shadow value)

The previous record average worth of each parameter within a period of time, increasing the generalization of the model.

For all parameters: w, b

tensorflow study notes 2 (real)

Guess you like