[Deep learning] Basic deep learning model based on TensorFlow

cifar10 training data set download

Link: https://pan.baidu.com/s/1Qlp2G5xlECM6dyvUivWnFg
Extraction code: s32t

Front configuration

Introduce the tensorflow library, and other auxiliary library files. The installation method is pip3 install tensorflow numpy pickle. The detailed process is not described here.
Here, the training and test data set files are placed in the parent folder of the script, so CIFAR_DIR is assigned according to the actual situation, and this parameter will be referenced in the subsequent process.

import tensorflow.compat.v1 as tf
tf.disable_v2_behavior()
import os
import pickle
import numpy as np

CIFAR_DIR = "./../cifar-10-batches-py"
print(os.listdir(CIFAR_DIR))

Organize image training set and test set

Introduction to data set analytical functions

Before training the model, the most basic thing is to analyze the structure of the data set and the analytical method.
Define an auxiliary function in the training script. The function of this function is to open the file in the passed parameter and store it in the data variable in the form of bytes. There will be two sub-data sets in the parsed data table, one is named data, the other is named label, data is the single image information itself, and label is the category of the data, such as a car.

def load_data(filename):
    """read data from data file."""
    with open(filename, 'rb') as f:
        data = pickle.load(f, encoding='bytes')
        return data[b'data'], data[b'labels']

Let's run the function with actual data to visualize the data and see how the data is processed.
Take a look at the file organization of this training set. The first is 5 data_batch, that is, 5 data groups, and a test_batch is used to test the trained neurons. Let's do an experiment with data_batch_1 and test_batch, and parse some of the data.

The following is a test script to read the first data and the first label in data_batch_1.
Similarly, we also need to read the first data and the first label in test_batch.

import os
import pickle
import numpy as np

CIFAR_DIR = "./cifar-10-batches-py"
print(os.listdir(CIFAR_DIR))

def load_data(filename):
    """read data from data file."""
    with open(filename, 'rb') as f:
        data = pickle.load(f, encoding='bytes')
        print(data)#将读取到的生文件直接显示出来，事实证明这里面改是个超大的字典
        return data[b'data'], data[b'labels']

train_filename = os.path.join(CIFAR_DIR, 'data_batch_1')
test_filename = os.path.join(CIFAR_DIR, 'test_batch')

train_data, train_labels = load_data(train_filename)
test_data, test_labels = load_data(test_filename)

print('data_batch_1 的数据个数：{}'.format(len(train_data)))
print('data_batch_1 的标签个数：{}'.format(len(train_labels)))
print('-'*30)
print('test_batch 的数据个数：{}'.format(len(test_data)))
print('test_batch 的标签个数：{}'.format(len(test_labels)))
print('-'*30)
print('train_batch_1 图片的像素：{}'.format(len(train_data[0])))
print('test_batch 图片的像素：{}'.format(len(test_data[0])))
print('-'*30)
print('第一个训练集数据及其标签')
print(train_labels[0])
print(train_data[0])
print('-'*30)
print('第一个测试集数据及其标签')
print(test_labels[0])
print(test_data[0])

First look at the unprocessed raw data set. It is a dictionary that contains the information of the group, labels, data and the name of each file. Each list in the data actually represents a pixel, representing different colors. All the pixels in this list form a small picture. For example, the information in the first data is shown in the list [59 , 43, 50, …, 140, 84, 72]. Let's see how long this list is.

Next, we count the specific information of this data set. The print information is as follows:

So you can see that there are 10,000 pictures in each batch, and the number of pixels in each picture is 3072=1024*3. We need to clarify here because the pictures are all It is composed of three primary colors RGB, so the picture is actually composed of 1024 (32 x 32) pixels, and 3 times the amount of data is caused by color information.

Finishing the data set

The function of the code is written in the comment after each line.
The main function of this class is to merge the data in all files into the same pair of lists. It
also provides the _shuffle_data function. If necessary, you can enable this function. , Under the premise of not destroying the corresponding relationship between the image data and the corresponding label, the order of all the list contents is disrupted, thereby improving the effectiveness of training.
Finally, this class also provides a function next_batch, which allows training and testing at a specific step value during training or testing, so that we can see the phased test results.

# tensorflow.Dataset.
class CifarData:
    def __init__(self, filenames, need_shuffle):#初始化函数，需要一个文件名称，和一个是否需要打乱顺序的置位符（选1则打乱）
        all_data = []#将所有图片数据放在这个列表中
        all_labels = []#将所有图片对应的标签放在这个列表中
        for filename in filenames:#例如，训练数据集中有5个文件，则依次循环这5个文件
            data, labels = load_data(filename)#在这里引用了上一节提供的文件加载函数，返回一个数据集 和 一个标签集
            all_data.append(data)#将所有5个文件中的图片数据添加到all_data列表中
            all_labels.append(labels)#将所有5个文件中的标签数据添加到all_labels列表中
        self._data = np.vstack(all_data)
        self._data = self._data / 127.5 - 1
        self._labels = np.hstack(all_labels)
        print(self._data.shape)
        print(self._labels.shape)
        
        self._num_examples = self._data.shape[0]#data shape是(50000,3702),所以self._data.shape[0]就代表50000，
        										#也就是说，_num_examples代表所有训练数据的个数
        self._need_shuffle = need_shuffle#如果置1，则启动打乱程序：_shuffle_data
        self._indicator = 0#这个值的作用是一个游标，每次执行完函数next_batch后，这个值就更新为当前所在的self._data列表的位置
        				#这很重要，因为我们需要判断50000个数据是否都训练到了，如果想再学习一轮以加固训练效果，则需要该游标进行判断
        				#具体的执行过程需要参考 next_batch函数 以及 训练和测试 章节
        if self._need_shuffle:
            self._shuffle_data()#这里需要注意，我们要对实例化后的数据集进行_shuffle_data
            
    def _shuffle_data(self):
        # [0,1,2,3,4,5] -> [5,3,2,4,0,1]
        p = np.random.permutation(self._num_examples)
        self._data = self._data[p]
        self._labels = self._labels[p]
    
    def next_batch(self, batch_size):
        """return batch_size examples as a batch."""
        end_indicator = self._indicator + batch_size#该示例的batch_size为20，也就是每次训练取列表中的20个图片数据，
        											#然而我们有50000个图片，这样的重复需要50000/20次，而每次完成训练后，把本实例中的self._indicator更新为本次训练的end_indicator 
		"""
		如果没啥幺蛾子，下面两段IF是不执行的。需要执行的话有两种可能：
		1. 训练数据集用完了，判断是否需要第二轮，或更多轮的训练
		2. 初始状态下就异常，八成是batch_size设置得太大
		剩余的数据不够batch_size个了，则直接放弃剩余数据，打乱顺序继续第二轮训练
		"""
        if end_indicator > self._num_examples:#如果本次训练end_indicator数值超过50000了，则
        									#说明我们学完了所有50000个图片，需要进行第二轮的学习
            if self._need_shuffle:
                self._shuffle_data()#第二轮学习为了有效，我们再次打乱50000个图片数据
                self._indicator = 0#游标再次置0，这样我们就可以愉快地开始第二轮训练了
                end_indicator = batch_size
            else:#如果当时初始化没有要求打乱，则证明我们不需要第二轮学习。这时中断测试，并抛出异常提示没有更多的训练数据了。
                raise Exception("have no more examples")#不用再看了，训练结束了，放学！
        if end_indicator > self._num_examples:#上一个IF语句中重置了end_indicator为batch_size值，
        									#如果此时还是比_num_examples大的话，则证明这个batch_size只是单纯地设置得太大了。
            raise Exception("batch size is larger than all examples")#抛出异常，中断测试，回去修改batch_size去...
        batch_data = self._data[self._indicator: end_indicator]#获取本次需要的20个图片数据，每次循环都会往后移动20个
        batch_labels = self._labels[self._indicator: end_indicator]#获取本次需要的20个标签数据，每次循环都会往后移动20个
        self._indicator = end_indicator#循环由外部引用来进行，每完成一次next_batch就更新一下self._indicator
        return batch_data, batch_labels#当然这个函数执行完就要交货了，也就是需要用于训练的20个数据，上面的一堆步骤都是前期审核和位置记录

train_filenames = [os.path.join(CIFAR_DIR, 'data_batch_%d' % i) for i in range(1, 6)]
test_filenames = [os.path.join(CIFAR_DIR, 'test_batch')]

train_data = CifarData(train_filenames, True)
test_data = CifarData(test_filenames, False)

Create neurons

x = tf.placeholder(tf.float32, [None, 3072])
# [None], eg: [0,5,6,3]
y = tf.placeholder(tf.int64, [None])

# (3072, 10)
w = tf.get_variable('w', [x.get_shape()[-1], 10],
                   initializer=tf.random_normal_initializer(0, 1))
# (10, )
b = tf.get_variable('b', [10],
                   initializer=tf.constant_initializer(0.0))

# [None, 3072] * [3072, 10] = [None, 10]
y_ = tf.matmul(x, w) + b

# mean square loss
"""
# course: 1 + e^x
# api: e^x / sum(e^x)
# [[0.01, 0.9, ..., 0.03], []]
p_y = tf.nn.softmax(y_)
# 5 -> [0,0,0,0,0,1,0,0,0,0]
y_one_hot = tf.one_hot(y, 10, dtype=tf.float32)
loss = tf.reduce_mean(tf.square(y_one_hot - p_y))
"""

loss = tf.losses.sparse_softmax_cross_entropy(labels=y, logits=y_)
# y_ -> sofmax
# y -> one_hot
# loss = ylogy_



# indices
predict = tf.argmax(y_, 1)
# [1,0,1,1,1,0,0,0]
correct_prediction = tf.equal(predict, y)
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float64))

with tf.name_scope('train_op'):
    train_op = tf.train.AdamOptimizer(1e-3).minimize(loss)

Training and testing

init = tf.global_variables_initializer()
batch_size = 20
train_steps = 100000
test_steps = 100

# run 100k: 30.95%
with tf.Session() as sess:
    sess.run(init)
    for i in range(train_steps):
        batch_data, batch_labels = train_data.next_batch(batch_size)
        loss_val, acc_val, _ = sess.run(
            [loss, accuracy, train_op],
            feed_dict={
    
    
                x: batch_data,
                y: batch_labels})
        if (i+1) % 500 == 0:
            print('[Train] Step: %d, loss: %4.5f, acc: %4.5f' 
                  % (i+1, loss_val, acc_val))
        if (i+1) % 5000 == 0:
            test_data = CifarData(test_filenames, False)
            all_test_acc_val = []
            for j in range(test_steps):
                test_batch_data, test_batch_labels \
                    = test_data.next_batch(batch_size)
                test_acc_val = sess.run(
                    [accuracy],
                    feed_dict = {
    
    
                        x: test_batch_data, 
                        y: test_batch_labels
                    })
                all_test_acc_val.append(test_acc_val)
            test_acc = np.mean(all_test_acc_val)
            print('[Test ] Step: %d, acc: %4.5f' % (i+1, test_acc))