Use python's plot to draw loss and acc curves and store them as pictures

Use python's plot to draw the loss curve and accuracy rate change curve during the network training process. The main idea here is to save the desired loss value and accuracy rate value first, save them in a .txt file, and wait for the network training to end , we then use the stored data to draw various curves.

The general steps are: data reading and storage -> loss curve drawing -> accuracy rate curve drawing

1. Data reading and storage part

We first need to get the data during training. Taking the loss value as an example, each iteration of the network will generate a corresponding loss, then we will store each loss value, store it in a list, and save it in a .txt file.

1.3817585706710815, 
1.8422836065292358, 
1.1619832515716553, 
0.5217241644859314, 
0.5221078991889954, 
1.3544578552246094, 
1.3334463834762573, 
1.3866571187973022, 
0.7603049278259277

The above picture shows some loss values, which vary according to the number of iterations. If it is iterated 10,000 times, there will be 10,000 loss values ​​here.
The accuracy value is a value generated by each epoch. If you train 100 epochs, there will be 100 accuracy values.

How is the loss value here saved to the file? First, find the network training code, which is main.py or train.py in the project. Find the training part first in the file. There is often such a line of code in it:

for epoch in range(resume_epoch, num_epochs):   # 就是这一行
	####
	...
	loss = criterion(outputs, labels.long())              # 损失样例
	...
    epoch_acc = running_corrects.double() / trainval_sizes[phase]    # 准确率样例
    ...
    ###

Starting from this line is the training part, and you will find these two similar codes below, which are the loss value and the accuracy rate value.

At this time, add the following code to the source code:

train_loss = []
train_acc = []
for epoch in range(resume_epoch, num_epochs):          # 就是这一行
	###
	...
	loss = criterion(outputs, labels.long())           # 损失样例
	train_loss.append(loss.item())                     # 损失加入到列表中
	...
	epoch_acc = running_corrects.double() / trainval_sizes[phase]    # 准确率样例
	train_acc.append(epoch_acc.item())                 # 准确率加入到列表中
	... 
with open("./train_loss.txt", 'w') as train_los:
    train_los.write(str(train_loss))

with open("./train_acc.txt", 'w') as train_ac:
     train_ac.write(str(train_acc))

In this way, the data storage of loss value and accuracy rate value is completed!

2. Draw the loss curve

Mainly requires numpy library and matplotlib library.

pip install numpy malplotlib

First, read the stored data in the .txt file, the following is the read function:

import numpy as np

# 读取存储为txt文件的数据
def data_read(dir_path):
    with open(dir_path, "r") as f:
        raw_data = f.read()
        data = raw_data[1:-1].split(", ")   # [-1:1]是为了去除文件中的前后中括号"[]"

    return np.asfarray(data, float)

Then, it is to draw the loss curve part:

if __name__ == "__main__":

	train_loss_path = r"/train_loss.txt"   # 存储文件路径
	
	y_train_loss = data_read(train_loss_path)        # loss值，即y轴
	x_train_loss = range(len(y_train_loss))			 # loss的数量，即x轴

	plt.figure()

    # 去除顶部和右边框框
    ax = plt.axes()
    ax.spines['top'].set_visible(False)
    ax.spines['right'].set_visible(False)

    plt.xlabel('iters')    # x轴标签
    plt.ylabel('loss')     # y轴标签
	
	# 以x_train_loss为横坐标，y_train_loss为纵坐标，曲线宽度为1，实线，增加标签，训练损失，
	# 默认颜色，如果想更改颜色，可以增加参数color='red',这是红色。
    plt.plot(x_train_loss, y_train_loss, linewidth=1, linestyle="solid", label="train loss")
    plt.legend()
    plt.title('Loss curve')
    plt.show()
    pit.savefig("loss.png")

In this way, the loss image is drawn! as follows:

3. Draw the accuracy curve
With the above foundation, it is much simpler.
Just one thing to remember, the x-axis above is the number of iterations, here is the training round epoch.

if __name__ == "__main__":

	train_acc_path = r"/train_acc.txt"   # 存储文件路径
	
	y_train_acc = data_read(train_acc_path)       # 训练准确率值，即y轴
	x_train_acc = range(len(y_train_acc))			 # 训练阶段准确率的数量，即x轴

	plt.figure()

    # 去除顶部和右边框框
    ax = plt.axes()
    ax.spines['top'].set_visible(False)
    ax.spines['right'].set_visible(False)

    plt.xlabel('epochs')    # x轴标签
    plt.ylabel('accuracy')     # y轴标签
	
	# 以x_train_acc为横坐标，y_train_acc为纵坐标，曲线宽度为1，实线，增加标签，训练损失，
	# 增加参数color='red',这是红色。
    plt.plot(x_train_acc, y_train_acc, color='red',linewidth=1, linestyle="solid", label="train acc")
    plt.legend()
    plt.title('Accuracy curve')
    plt.show()
    pit.savefig("acc.png")

In this way, the accuracy change curve is drawn! as follows:

The following is the complete code, taking drawing the accuracy curve as an example, and changing the x-axis to iters, consistent with the loss curve, for reference:

import numpy as np
import matplotlib.pyplot as plt


# 读取存储为txt文件的数据
def data_read(dir_path):
    with open(dir_path, "r") as f:
        raw_data = f.read()
        data = raw_data[1:-1].split(", ")

    return np.asfarray(data, float)


# 不同长度数据，统一为一个标准，倍乘x轴
def multiple_equal(x, y):
    x_len = len(x)
    y_len = len(y)
    times = x_len/y_len
    y_times = [i * times for i in y]
    return y_times


if __name__ == "__main__":

    train_loss_path = r"/train_loss.txt"
    train_acc_path = r"/train_acc.txt"

    y_train_loss = data_read(train_loss_path)
    y_train_acc = data_read(train_acc_path)

    x_train_loss = range(len(y_train_loss))
    x_train_acc = multiple_equal(x_train_loss, range(len(y_train_acc)))

    plt.figure()

    # 去除顶部和右边框框
    ax = plt.axes()
    ax.spines['top'].set_visible(False)
    ax.spines['right'].set_visible(False)

    plt.xlabel('iters')
    plt.ylabel('accuracy')

    # plt.plot(x_train_loss, y_train_loss, linewidth=1, linestyle="solid", label="train loss")
    plt.plot(x_train_acc, y_train_acc,  color='red', linestyle="solid", label="train accuracy")
    plt.legend()

    plt.title('Accuracy curve')
    plt.show()
    pit.savefig("acc.png")