Tensorflow⑤——Build a neural network framework with keras

tf.keras is a highly encapsulated framework introduced by tensorflow2, which can be used to quickly build neural network models. Keras was born to support rapid experiments and can quickly convert ideas into results. It is the ultimate easy-to-use deep learning framework. It provides a consistent and concise API, which can greatly reduce the workload of general applications and improve code encapsulation and reusability.

Keras official documentation:
https://tensorflow.google.cn/api_docs/python/tf

Six methods of building a neural network

tf.keras builds six methods of neural network

import
train, test
model = tf.keras.models.Sequential / class MyModel(Model) model=MyModel
model.compile
model.fit
model.summary

The first step : import related modules, such as import tensorflow as tf.
Step 2 : Specify the training set and test set of the input network, such as specifying the input x_train and label y_train of the training set, and the input x_test and label y_test of the test set.
Step 3 : Build the network structure layer by layer, model = tf.keras.models.Sequential(). Or use the method of inheriting
the fourth step : configure the training method in model.compile(), and select the optimizer, loss function and final evaluation index used during training.
Step 5 : Execute the training process in model.fit(), and inform the input values ​​and labels of the training set and test set, the size of each batch (batchsize) and the number of iterations of the data set (epoch).
Step 6 : Use model.summary() to print the network structure and count the number of parameters.

Introduction to function usage

tf.keras.models.Sequential()

The Sequential function is a container that describes the network structure of the neural network, and the network structure from the input layer to the output layer is described in the input parameters of the Sequential function.
like:

Straighten layers : tf.keras.layers.Flatten()

• The straightening layer can transform the size of the tensor and straighten the input features into a one-dimensional array, which is a layer without calculation parameters.

---

Fully connected layer :

tf.keras.layers.Dense( 神经元个数,
					   activation=”激活函数”,
                       kernel_regularizer=”正则化方式”)

in:

• activation (string given) optional relu, softmax, sigmoid, tanh, etc.
• kernel_regularizer 可选 tf.keras.regularizers.l1()、tf.keras.regularizers.l2()

Convolution layer :

tf.keras.layers.Conv2D( filter = 卷积核个数,
						kernel_size = 卷积核尺寸,
 					    strides = 卷积步长,
                        padding = “valid” or “same”)

LSTM layer:

tf.keras.layers.LSTM() 

Commonly used such as:

model = tf.keras.models.Sequential([
	# tf.keras.layers.Dense(神经元个数, activation='激活函数', kernel_regularizer=正则化方式)
    tf.keras.layers.Dense(3, activation='softmax', kernel_regularizer=tf.keras.regularizers.l2)
])

model.compile

 model.compile( optimizer = 优化器, loss = 损失函数, metrics = [“准确率”])

Compile is used to configure the training method of the neural network, and inform the optimizer, loss function and accuracy evaluation standard used during training.
Among them :

• optimizer can be the name of the optimizer given in the form of a string, or it can be in the form of a function, and the learning rate, momentum and hyperparameters can be set using the function form.
  Options include:

   ‘Sgd’or tf.keras.optimizers.SGD( lr=学习率,
      						  decay=学习率衰减率,
							  momentum=动量参数)
							  
   ‘Adagrad’or tf.keras.optimizers.Adagrad(lr=学习率,
							  			   decay=学习率衰减率)
							  			   
   ‘Adadelta’or tf.keras.optimizers.Adadelta(lr=学习率,
											 decay=学习率衰减率)
											 
   ‘Adam’or tf.keras.optimizers.Adam (lr=学习率,
									  decay=学习率衰减率)

• loss can be the name of the loss function given as a string or as a function.
  Options include:

   ‘mse’or tf.keras.losses.MeanSquaredError()
   ‘sparse_categorical_crossentropy or tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False)
   # 损失函数常需要经过 softmax 等函数将输出转化为概率分布的形式。
   # from_logits 则用来标注该损失函数是否需要转换为概率的形式，取 False 时表示已经转化为概率分布，取 True 时表示没有转化为概率分布，直接输出。

• Metrics mark network evaluation indicators.
  Options include:

‘accuracy’：y_和 y 都是数值，如 y_=[1] y=[1]。

‘categorical_accuracy’：y_和 y 都是以独热码和概率分布表示。
	如 y_=[0, 1, 0], y=[0.256, 0.695, 0.048]
	
‘sparse_categorical_accuracy’：y_是以数值形式给出，y 是以独热码形式给出。
	如 y_=[1],y=[0.256, 0.695, 0.048]。

model.fit: used to perform the training process

model.fit(训练集的输入特征， 训练集的标签， batch_size, epochs,
 		  validation_data = (测试集的输入特征，测试集的标签)，
 		  validataion_split = 从测试集划分多少比例给训练集，
 		  validation_freq = 测试的 epoch 间隔次数) 

model.summary()

The summary function is used to print the network structure and parameter statistics

example

• Take the iris data set in sklearn as an example, use the Sequential function to initialize, and use the 6-step method for accuracy recognition

import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
import tensorflow as tf
import numpy as np
from sklearn import datasets

x_train = datasets.load_iris().data
y_train = datasets.load_iris().target

np.random.seed(116)
np.random.shuffle(x_train)
np.random.seed(116)
np.random.shuffle(y_train)
tf.random.set_seed(116)

model = tf.keras.models.Sequential([
# tf.keras.layers.Dense( 神经元个数, activation='激活函数', kernel_regularizer=正则化方式 )
    tf.keras.layers.Dense(3, activation='softmax', kernel_regularizer=tf.keras.regularizers.l2())
])

# Model.compile( optimizer = 优化器, loss = 损失函数, metrics = ['准确率'])
model.compile(
    # optimizer=tf.keras.optimizers.SGD(learning_rate=0.1),
    optimizer=tf.keras.optimizers.SGD(lr=0.1),
    loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
    metrics=['sparse_categorical_accuracy']
)

# model.fit(训练集的输入特征， 训练集的标签， batch_size, epochs,
#  		    validation_data = (测试集的输入特征，测试集的标签)，
#  		    validataion_split = 从测试集划分多少比例给训练集，
#  		    validation_freq = 测试的 epoch 间隔次数)
model.fit(x_train, y_train, batch_size=32, epochs=500, validation_split=0.2, validation_freq=20)

model.summary()

• Take the iris data set in sklearn as an example, use the class function to initialize, and use the 6-step method for accuracy recognition

import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
import tensorflow as tf
from tensorflow.keras.layers import Dense
from tensorflow.keras import Model
from sklearn import datasets
import numpy as np

x_train = datasets.load_iris().data
y_train = datasets.load_iris().target

np.random.seed(116)
np.random.shuffle(x_train)
np.random.seed(116)
np.random.shuffle(y_train)
tf.random.set_seed(116)

class IrisModel(Model):
    def __init__(self):
        super(IrisModel, self).__init__()
        self.d1 = Dense(3, activation='softmax', kernel_regularizer=tf.keras.regularizers.l2())

    def call(self, x, training=None, mask=None):
    # def call(self, x):
        y = self.d1(x)
        return y

model = IrisModel()

model.compile(optimizer=tf.keras.optimizers.SGD(lr=0.1),
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=False),
              metrics=['sparse_categorical_accuracy'])

model.fit(x_train, y_train, batch_size=32, epochs=500, validation_split=0.2, validation_freq=20)
model.summary()