PyTorch- training Visualization

Visualization Training

Brief introduction

The depth of the neural network is a complex mathematical model, which can be a long time for people interpretative question, known as the "black box" model. But its essence is a mathematical model, a lot of statistical methods can be used to observe the depth of understanding of this type of model. And no built-in PyTorch perfect visualization capabilities, usually by means of TensorBoard TensorFlow visualized (TensorBoardX use this tool) or use Visdom this visualization tool, these two methods is more mainstream means.

TensoBoardX

Training process typically neural network is long and complicated, visual training process determines for discovery and convergence of the model performance issues are very important and need to visualize typically consists Loss curve, the Accuracy curves, wherein FIG visualization, prediction confusion matrix and the like.

TensorBoardX package is to allow outside TensorFlow framework can be used for training TensorBoard visualization, which official documents can look yourself.

installation

Pip use tools to install the package. (Under the premise of the installation tensorboard, use the command pip install tensorboardxto install it.)

Enable Monitoring

TensorBoard visualization of data from the local log file exists in a folder and specify the folder to monitor the folder open TensorBoard service console.

Enable tensorboard service in the virtual environment, enable command tensorboard --logdir=logs(already created a logsfolder), open the default port 6006 to provide services, by http://localhost:6006/the results can be accessed visualization.

API function interface

General need to build a global event writer (event writer), and then by calling the writer object add_xadd the relevant data, images and other methods to build writer objects to monitor the incoming directory can be referred to herein logsfolder.

Github end of this section gives all test code files.

• writer.add_scalar(tag, scalar_value, global_step=None, walltime=None)
  • Recording a scalar changes in a diagram, curve drawing Loss commonly used.
  • tag title parameter for the graph.
  • scalar_value parameter indicates the value of the step, as the ordinate.
  • global_step parameter indicates the number of steps, as abscissa.
  • walltime parameter indicates the file name for the event set-up time, the current default time.
    Such as drawing a linear function of the attenuation loss as FIG.

• writer.add_scalars(main_tag, tag_scalar_dict, global_step=None, walltime=None)
  • Recording changes in a plurality of scalar graph, commonly used in Comparative training set and test set loss loss.
  • main_tag parameter sets the label of FIG.
  • tag_scalar_dict parameter represents the form of a plurality of scalar dictionaries, key as a tag, value of a scalar value, which are the ordinate.
  • Other parameters above.
    A change in the image drawing, for example, training and validation sets loss follows.

• writer.add_histogram(tag, values, global_step=None, bins='tensorflow', walltime=None)
  • Histograms and quantile draw line chart for detecting a change in the distribution of weights and the gradient of the network updated viewing direction is correct.
  • values parameter indicates the value of the histogram drawing.
  • bins parameters to determine how to take the bins.
  • The remaining parameters above.

FIG model parameters such as drawing, attention to the need to toggle the top bar to another view, such HISTOGRAMS.

• writer.add_image(tag, img_tensor, global_step=0, walltime=0)
  • Drawn picture, commonly inputted to the detection pattern, wherein FIG, weight, etc., but the interface can only be a visual image, it is usually by means of torchvision.utils.make_grid()a plurality of pictures as a mosaic picture, which follows API, specifically to view official documents, torchvision.utils.make_grid(tensor, nrow=8, padding=2, normalize=False, ra nge=None, scale_each=False, pad_value=0).
  • img_tensor parameter indicates the need to visualize the image data required (C, H, W)format.
  • The remaining parameters above.
    The figure is the result of more than random noise of the picture, also need the top navigation bar to switch view.

• writer.add_graph(model, input_to_model=None, verbose=False)
  • Draw Depth neural network topology.
  • model参数表示模型实例。
  • input_to_model参数表示模型的输入数据，给一个随机数符合shape要求即可。
    生成的是一个可编辑的模型可视化结果。

• writer.add_embedding(mat, metadata=None, label_img=None, global_step=0, tag='default', metadata_header=None)
  • 在三维或者二维空间内展示数据分布，可选T-SNE、PCA或者Custom方法。
  • mat参数表示需要绘制的数据，一个样本必须是一个向量，即为(N,D)维度。
  • metadata参数表示数据的标签，为一个长度为N的列表。
  • label_img参数表示空间中展示的图片，shape为(N,C,H,W)。
  • 其他参数同上。
    可视化MNIST数据集结果如下。

• add_text(tag, text_string, global_step=None, walltime=None)
  • 记录文字信息。
• add_text(tag, text_string, global_step=None, walltime=None)
  • 记录视频信息。
• add_figure(tag, figure, global_step=None, close=True, walltime=None)
  • 添加plt的图片到图像中。
• add_figure(tag, figure, global_step=None, close=True, walltime=None)
  • 在图像中绘制边界框，常用于目标检测。
• add_figure(tag, figure, global_step=None, close=True, walltime=None)
  • 绘制PR曲线。
• export_scalars_to_json(path)
  • 导出scalars信息为json文件，方便后续使用。

上面就是TensorBoardX常用的一些接口，通过这些接口可以实现很多复杂的可视化操作，如卷积核可视化、特征图可视化、梯度分布可视化等等，这里就不提及了，后面的实战文章我会用到。

补充说明

本文介绍了TensorBoardX这个训练可视化工具，这是非常实用的适合PyTorch使用者的工具，当训练遇到问题时方便的可视化会很容易判断出模型的问题，从而优化得到较好的模型设计。本文涉及到的代码均可以在我的Github找到，欢迎star或者fork。