TensorFlow linear regression case
To tell the story through a case study of how to do linear regression using tensorflow
What is the linear regression
Linear regression equation:
1 wherein: y = w * x + b, x is the input, y is the output, w is a weight, b is the bias
n feature: y = w1 * x1 + w2 * x2 ...... + wn * xn + b, n-weights, a bias
Linear regression of the core:
- Algorithms: Linear Regression
- Strategy: mean square error
- Optimization: gradient descent, need to set the learning rate
The step of using linear regression
1. Prepare characteristics and target values, i.e. data, wherein the input value is the target value is output to the predicted rate, for example,
Location, size, orientation, etc. are characteristic values, the target value is the total price of the room.
2. The model requires a weight, an offset, and then random initialization, to obtain a prediction function y_predict.
3. seeking loss function, i.e. error, here is the mean square error.
4. loss of optimization, linear regression, we optimize loss by way of gradient descent, minimum loss function,
Figurative saying that have given a random point, the process of finding a base of the mountains from the hill, as shown in FIG.
Description: What is the mean square error?
Is the predicted value and the target value and differencing square root, and then averaged, the following formula:
[(Y1-y1 ') ^ 2 + .... + (as-is') ^ 2] / n
The core api
Gradient descent optimization API: tf.train.GradientDescentOptimizer (learning_rate)
Parameter Description:
learning_rate: learning rate, typically of the order of 0.1, 0.01 and so on, of course, can be 1,10,100, but the basic is not so written.
Function returns a gradient descent op, we can be in the session.
Core code
. 1 Import tensorflow TF AS
2 Import OS
. 3
. 4
. 5 DEF mymigration ():
. 6 "" "
. 7 from a linear regression prediction achieved
. 8 : return:
. 9 " ""
10 # 1. preparing data, here we simulate data via a formula, If the data is true, we can read the external data
. 11 X = tf.random_normal ([100,1], Mean = 0.0, STDDEV = 1.0 )
12 is # matrix multiplication must be two-dimensional
13 is y_true tf.matmul = ( X, [[0.7]]) + 0.8
14
15
16 # 2. establish a linear regression model, a weight, an offset, Y = W * X + B
. 17 # random weights and to a bias, so that he to calculate the loss, which is equivalent to the starting point of the gradient descent
18 #Here, our weight and bias is the need to optimize, he is change, must be used to define tf.Variable
19 # in process definition, we can set the stage trainable = False to restrict a variable does not change with the gradient descent be optimized default True
20 is weight = tf.Variable (tf.random_normal ([1,1], Mean = 0.0, STDDEV = 1.0, name = ' W ' ))
21 is BIAS = tf.Variable (0.0, name = ' B ' )
22 is y_prerdict tf.matmul = (X, weight) + BIAS
23 is
24
25 # to establish a loss function, the mean square error, tf.square seek error, reduce_mean averaging
26 is loss = tf.reduce_mean (tf.square ( y_true- y_prerdict))
27
28
29 # gradient descent optimization loss, learning rate is 0.1.
30 = tf.train.GradientDescentOptimizer train_op (0.1 ) .minimize (Loss)
31 is
32
33 is # OP define a variable initialization
34 is init_op = tf.global_variables_initializer ()
35 # is run by a program session
36 with tf.Session () AS Sess:
37 # initialize variables
38 is sess.run (init_op)
39 # operation optimization 200, i.e. run op 200 times
40 for I in Range (200 ):
41 is sess.run (train_op)
42 is Print ( ' % d of suboptimal and bias weights: weight is:% f, bias: F% ' %(i,weight.eval(),bias.eval()))
43 return None
44 if __name__ == "__main__":
45 mymigration( )
Run results are shown below:
Because there is no data, by the equation I y_true = tf.matmul (x, [[ 0.7]]) + 0.8 random sample data
In practice, we can read external data, and build model train
200 words by training times, we finally hit the right get a weight of 0.7, 0.8, training more fit data models can be directly and analog data
Comparison formula, if we are to import external data, it is not know good or bad training model, we need to constantly improve the optimization model of quasi
Probability.
Learning rate
learn__rate gradient descent is a very important parameter, he decided to step from the top to reach the base of the mountains and speed, they are generally small, 0.1,
0.01, so there.
Learn _rate too big too small is not good, is too large, then gradient may cause an explosion, is too small will not reach the desired training effect, the need to increase the number of training
Additional information:
Gradient explosion: In extreme cases, the value of the weight becomes so large that an overflow, leading to NAN value but is gradient explosion, the explosion in the learning problems gradient
The depth of the neural network RNN more prone.
How to solve gradient explosion: (1): redesigned network, (2) to adjust the learning rate, (3) using a gradient truncated, check and limit the gradient in the training process of
Size; (4) using the activation function