Previous section: Linear regression from scratch
Use the keras interface recommended by tensorflow2.0 to implement linear regression training.
1. Generate a data set
Use the same data set as in the previous section. Among them featuresare the training data features and labelsthe labels.
import tensorflow as tf
num_inputs = 2
num_examples = 1000
true_w = [2, -3.4]
true_b = 4.2
features = tf.random.normal(shape=(num_examples, num_inputs), stddev=1)
labels = true_w[0] * features[:, 0] + true_w[1] * features[:, 1] + true_b
labels += tf.random.normal(shape=(labels.shape), stddev=0.01)
2. Read the data
Use the data module in tensorflow to read data
from tensorflow import data as tfdata
batch_size = 10
#将训练数据的特征和标签组合
dataset = tfdata.Dataset.from_tensor_slices((features, labels))
#随机读取小批量
dataset = dataset.shuffle(buffer_size=num_examples)
dataset = dataset.batch(batch_size)
data_iter = iter(dataset)
shuffleThe buffer_sizeparameter should be greater than the number of samples, and the split size batchcan be specified batch_size.
for X, y in data_iter:
print(X, y)
break
3 Define model and initialization parameters
To use the kerasdefined network, first define a model variable model, which is an Sequentialexample.In Keras, the Sequentialinstance can be seen as a container connecting layers.
When constructing the model, we sequentially add layers to the container. When given input data, each layer in the container will infer the input size of the next layer in turn. The important point is that in keras, we don't need to specify the shape of each layer input. In linear regression, the input layer and output layer are equivalent to a fully connected layer keras.layers.Dense().
kerasThe initialization parameters kernel_initializerand bias_initializeroptions are used to set the initialization mode of weight and bias respectively. We tensorflowimport the initializersmodule,Each element of the specified weight parameter will be randomly sampled from a normal distribution with a mean of 0 and a standard deviation of 0.01 during initialization. The deviation parameter is initialized to 0 by default.RandomNormal(stddev=0.01)Each element of the specified weight parameter will be randomly sampled from a normal distribution with a mean of 0 and a standard deviation of 0.01 during initialization. The deviation parameter is initialized to 0 by default.
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow import initializers as init
model = keras.Sequential()
model.add(layers.Dense(1, kernel_initializer=init.RandomNormal(stddev=0.01)))
4 Define the loss function
TensorflowThe lossesmodule provides base classes of various loss functions and custom loss functions, and directly uses its mean square error loss as the loss function of the model.
from tensorflow import losses
loss = losses.MeanSqyaredError()
5 Define the optimization algorithm
tensorflow.keras.optimizersThe module provides many commonly used optimization algorithms such as SGD, Adam and RMSProp. Next, create an optimizer instance for optimizing all parameters of the model, and specify a small batch stochastic gradient descent (SGD) with a learning rate of 0.03 as the optimization algorithm.
6 training model
When using the Tensorflowtraining model, we tensorflow.GradientTaperecord the dynamic graph gradient by calling . By model.trainable_variablesfinding the variables that need to be updated and using the trainer.apply_gradientsupdated weights, one step of training is completed.
num_epochs = 3
for epoch in range(1, num_epochs + 1):
for (batch, (X, y)) in enumrate(dataset):
with tf.GradientTape() as tape:
l = loss(model(X, training=True), y)
grads = tape.gradient(l, model.trainable_variables)
trainer.apply_gradients(zip(grads, model.trainable_variables))
l = loss(model(features), labels)
print('epoch %d, loss: %f' % (epoch, l))
Output:
epoch 1, loss: 0.519287
epoch 2, loss: 0.008997
epoch 3, loss: 0.000261
Compare the learned model parameters with the real model parameters.
Use the model get_weights()to get its weight ( weights) and deviation ( bias).
true_w, model.get_weights()[0]
true_b, model.get_weights()[1]
Output:
([2, -3.4], array([[ 1.9930198],
[-3.3977082]], dtype=float32))
(4.2, array([4.1895046], dtype=float32))