Python- deep learning - learning Notes (13): keras build convolutional neural network (two-dimensional data in a one-dimensional convolution)

Python- deep learning - learning Notes (13): keras build convolutional neural network (two-dimensional data in a one-dimensional convolution)

Convolutional neural network learning image classification is a depth of image processing applications, the advantages of the convolutional neural network can be directly convolved with the image pixels, image features are extracted from the image pixels, this approach closer to the human brain's visual treatment system. Further, the value of the right convolutional neural network and share pooling properties make the network layer parameters need to be trained greatly reduced, simplified network model to improve the efficiency of training.

A, networks built convolution step

1, model

1.1, build Sequential model

Sequential model is a linear stack of a plurality of neural networks. A similar frame structure, and then the design parameters and a good layer to layer is filled into the model, to achieve the effect of a whole network.

from keras.layers import Dense,Dropout,Flatten,Conv2D,MaxPooling2D
"""
	建立Sequential模型
"""
from keras.models import Sequential #在使用前需要先提前导入这个函数
model = Sequential()

1.2, the convolution build layer

Convolution convolutional neural network layer, also called feature extraction layers, comprising two parts. The first part is the convolution of the true level, the main role is to input data to extract features. Each feature different convolution kernels to extract input data are not the same, the greater the number of layers convolution kernel convolution, can be extracted more features of the input data.

"""
    建立卷积层1
    
    参数：
        filters - 建立的滤镜个数
        kernel_size - 滤镜大小
        padding - 填充：当为'same'时，使得图像卷积后的大小保持不变
        input_shape - 输入图形大小
        activation - 设置激活函数
"""
model.add(Conv2D(filters=16,
                 kernel_size=(5,5),
                 padding = 'same',
                 input_shape = (28,28,1),
                 activation = 'relu'))

1.3, pooled layer

pooling layer, also known as down-sampling layer, the main purpose is to reduce the amount of data processing on the basis of retaining useful information to speed up the training of the network speed. Normally, convolutional neural network convolution layer comprising at least two layers (here, the true convolution layer and the lower layer is referred to as sampled convolution layer), i.e. a convolution layer, Pooling layer, a convolution layer, Pooling layer. Convolution more layers can be extracted one more abstract features of the convolution on the basis of the previous layer.

"""
    建立池化层1
    
    参数：
        pool_size - 池化层大小
        
"""
model.add(MaxPooling2D(pool_size = (2,2)))

After that you can continue to add convolution Layer 2 -> 2 ...... pooling layer neural network based on your needs may be, we will not go into too much to do.

1.4, the flat layer to establish

Since it passed through the filter, so that the data changes dimension, and the role of this step is prior to input to the neural network, convert the data to be processed as one-dimensional vector corresponding to the input layer of the neural circuits.

#建立平坦层
model.add(Flatten())

1.5, establishing the hidden layer and output layer

This step is the establishment of a multi-layer neural network with the same method, it will not do too much explanation.
How to set up a blog on a single-layer or multi-layer neural networks see.

https://blog.csdn.net/qq_42826337/article/details/89341800

#建立隐蔽层
model.add(Dense(128,activation='relu'))
model.add(Dropout(0.5))

#建立输出层
model.add(Dense(10,activation='softmax'))

A plurality of layers may comprise fully connected, in fact, part of the hidden layer of a multilayer perceptron. Typically rear layer neural nodes in each case and the neural node of the previous layer is connected with a node between one layer of neurons is not connected. Each layer neuron nodes are connected to the front line weights for propagation, to obtain a weighted combination of the input layer neuron node through.

1.6 Model Summary View

print(model.summary())

2, start training

2.1, the definition of training methods

"""
    定义训练方式

    参数：
        loss - 损失函数： 这里采用交叉熵的方式
        optimizer - 优化器: 使用adam优化器可以让训练收敛更快
        metrics - 评估模型：设置为准确率

"""
model.compile(loss='categorical_crossentropy',
              optimizer='adam',metrics=['accuracy'])

2.2, start training

"""
    开始训练

    参数：
        x_train4D_normalize - feature数字图像的特征值
        y_trainOneHot - 数字图像的真实标签
        validation_spli - 训练与验证数据比例：80%用作训练数据，20%用作验证数据
        epochs - 训练周期
        batch_size - 每批次的数据项数
        verbose - 显示训练过程
"""
train_history=model.fit(x=x_train4D_normalize,
                        y=y_trainOneHot,validation_split=0.2,
                        epochs=10,batch_size=300,verbose=2)

2.3, draw accurate implementation process

import matplotlib.pyplot as plt
def show_train_history(train_history,train,validation):
    """
    显示训练过程

    参数：
        train_history - 训练结果存储的参数位置
        train - 训练数据的执行结果
        validation - 验证数据的执行结果
        
"""
    plt.plot(train_history.history[train])
    plt.plot(train_history.history[validation])
    plt.title('Train History')
    plt.ylabel(train)
    plt.xlabel('Epoch')
    plt.legend(['train','validation'],loc = 'upper left')
    plt.show()

show_train_history(train_history,'acc','val_acc') #绘制准确率执行曲线
show_train_history(train_history,'loss','val_loss') #绘制损失函数执行曲线

Because here the training set is always greater than the accuracy of the validation set accuracy, so we need to increase the training period or to increase the number of layers in order to improve the accuracy of concealment, it achieved under the premise of accuracy over the fitted curve does not happen as much as possible close. We can see that the value of the cost function convolution network drop very low, indicating that the convolution can increase the accuracy of the training, to train closer to the exact value of the parameter.

3, to assess the accuracy of the model

According to the training to analyze the accuracy of this model.

scores = model.evaluate(x_test4D_normalize,y_testOneHot)
scores[1]

4, confusion matrix display

Confusion matrix, also known as error matrix, is a specific form of display, allows us to have a visual way to understand supervised learning algorithm results, to see whether the model algorithm to confuse the two classes.

import pandas as pd
pd.crosstab(y_test,prediction,
            rownames=['label'],colnames=['predict'])

Diagonal to predict the correct number, other non-diagonal numbers represent a certain label prediction error will become another label.

Second, the two-dimensional data is one-dimensional convolution

We will encounter some one-dimensional data Under normal circumstances, such as natural language processing through signal after FFT convolution, or some audio signal, etc., to the convolution of this data, we will use a one-dimensional convolution Methods.

As shown, the left we need convoluted signal, a right filter, and the convolution method similar to two-dimensional convolution, is multiplied by -> addition -> ...... translation, thereby extracting features after the original value of the compressed data, thereby simplifying the network.

1, how to build a network of one-dimensional convolution

The method described above is 1-dimensional data for one-dimensional convolution is relatively simple, the focus here say how the two-dimensional and one-dimensional convolution of the above data.

1.1, introducing one-dimensional convolution function

First we have to pour a variety of functions for 1-dimensional convolution of the network.

from keras.models import Sequential
from keras.layers import Dense,Dropout,Flatten,Conv1D,MaxPooling1D

1.2, dimension reduction (for the two-dimensional data do dimensional convolution case 1)

Here to say that we in the pre-processing of data, if not one-dimensional but two-dimensional data, then, be sure to reduce the dimensionality of data, such as data a 28 * 28 before me, and we want to change it 28 * 28 = 784.

For clarity, here we make a comparison.

(1) a two-dimensional convolution (no data dimension reduction)

x_train4D = x_train.reshape(x_train.shape[0],28,28,1).astype('float32')
x_test4D = x_test.reshape(x_test.shape[0],28,28,1).astype('float32')
x_train4D_normalize = x_train4D / 255
x_test4D_normalize = x_test4D / 255

(2) a one-dimensional convolution (the need to reduce the dimensionality of the data)

x_train3D = x_train.reshape(x_train.shape[0],784,1).astype('float32')
x_test3D = x_test.reshape(x_test.shape[0],784,1).astype('float32')
x_train3D_normalize = x_train3D / 255
x_test3D_normalize = x_test3D / 255

Emphasize that (60000,28,28,1) final "1" is the convolution of the number of channels needed dimensions and filters, that is to say,Data last one is the number of channels, not the dimension data. You can not reduce the dimensionality of the data written in the dimension (60000,1,784), so that data becomes 60000, a data size, number of channels 784; you must be written (60000,784,1), represents 60,000 a data size 784, a number of channels.

1.3, model

After all the convolution layers and layers pool should call 1-dimensional convolution function, that is, before Conv2D have become Conv1D.

It should be said that even the one-dimensional convolution, we still need a flat layer, because after convolution, the number of channels also accounted for one dimension, and the whole, or a data cube, not as one-dimensional data into the input layer of the neural network, I think here for a long time, as shown:

this is a one-dimensional convolution network model summary, our input is the size of a 784-dimensional data, the number of the next layer of the filter is 10, and so on ... ...
Finally, before the flat layer, which is part of my circle out, is a two-dimensional data, you need to become a one-dimensional data 490 by a flat layer 49 * 10, to afferent input layer of the network.

After all the model methods are established with the same two-dimensional.

---

Attached below the two-dimensional data codes dimensional convolution of Case 1:

from keras.datasets import mnist
from keras.utils import np_utils
import numpy as np
from keras.models import Sequential
from keras.layers import Dense,Dropout,Flatten,Conv1D,MaxPooling1D
np.random.seed(10)


(x_train,y_train),(x_test,y_test) = mnist.load_data()
#将features转换为三维矩阵
x_train3D = x_train.reshape(x_train.shape[0],784,1).astype('float32')
x_test3D = x_test.reshape(x_test.shape[0],784,1).astype('float32')
#将feature标准化
x_train3D_normalize = x_train3D / 255
x_test3D_normalize = x_test3D / 255
#将label进行one hot转换
y_trainOneHot = np_utils.to_categorical(y_train)
y_testOneHot = np_utils.to_categorical(y_test)


#建立一个Sequential线性堆叠模型
model = Sequential()
model.add(Conv1D(filters=16,
                 kernel_size=25,
                 padding = 'same',
                 input_shape = (784,1),
                 activation = 'relu'))
model.add(MaxPooling1D(pool_size = 4))
#建立卷积层2
model.add(Conv1D(filters=10,
                 kernel_size=25,
                 padding = 'same',
                 activation = 'relu'))
#建立池化层2
model.add(MaxPooling1D(pool_size = 4))
model.add(Dropout(0.25))
#建立平坦层
model.add(Flatten())
#建立隐蔽层
model.add(Dense(128,activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(10,activation='softmax'))
print(model.summary())

model.compile(loss='categorical_crossentropy',
              optimizer='adam',metrics=['accuracy'])
train_history=model.fit(x=x_train3D_normalize,
                        y=y_trainOneHot,validation_split=0.2,
                        epochs=30,batch_size=300,verbose=2)
import matplotlib.pyplot as plt
def show_train_history(train_history,train,validation):
    """
    显示训练过程

    参数：
        train_history - 训练结果存储的参数位置
        train - 训练数据的执行结果
        validation - 验证数据的执行结果
        
"""
    plt.plot(train_history.history[train])
    plt.plot(train_history.history[validation])
    plt.title('Train History')
    plt.ylabel(train)
    plt.xlabel('Epoch')
    plt.legend(['train','validation'],loc = 'upper left')
    plt.show()
    
show_train_history(train_history,'acc','val_acc')
show_train_history(train_history,'loss','val_loss')
scores = model.evaluate(x_test3D_normalize,y_testOneHot)
scores[1]