Getting started with PyTorch (DataSet reading data and use of tensorboard)

---

Use python's own functions to view Pytorch modules

dir函数可以列出对象的模块标识符，标识符有函数、类和变量。

dir()Query the classes contained in the frame

Ctrl + FQuick query.

If you continue to query, you can query its submodules.


The two double dashes before and after refer to special variables.

helpWhat does function query function do?

DataSet: Provides a way to obtain data and its labels

The first way is to view the usage instructions of the classhelp(类)

The second way is to view the usage instructions of the class类??

DataSet reads data from NYUdepth V2 dataset

#coding=gbk         #指定编码格式为gbk
from  torch.utils.data import  Dataset
import cv2          #用于读取数据
import  os
class NYUDataSet(Dataset):
    def __init__(self):
        self.root_dir="D:\\AdeepLearningTest\\Code\\NYUDepthv2"
        self.rgb_path=os.path.join(self.root_dir,'RGB') #rgb图片的路径# coding=gbk
        self.label_path = os.path.join(self.root_dir, 'Label')  # 标签图片的路径
        self.allimg_path=os.listdir(self.rgb_path)
        self.allabel_path=os.listdir(self.label_path)
    def __getitem__(self, idx): #图像的下标也就是第几个
        img_name=self.allimg_path[idx]
        label_name = self.allabel_path[idx]
        label_item_path = os.path.join(self.label_path, label_name);
        img_item_path=os.path.join(self.rgb_path,img_name);
        img=cv2.imread(img_item_path)
        label=cv2.imread(label_item_path)
        return img ,label
    def __len__(self):
        return len(self.allimg_path)

#测试
data=NYUDataSet()
img,label=data[0]
print(len(data))
cv2.imshow("RGB",img)
cv2.imshow("Label",label)
cv2.waitKey()

Use of tensorboard

Install tensorboard

Instance classSummaryWriter

Parameter Description:

"""Creates a `SummaryWriter` that will write out events and summaries to the event file.
    
       类的参数说明 Args:（知道第一个参数一般就够了）
            log_dir (str): log_dir为输出日志的文件目录. Default is  runs/**CURRENT_DATETIME_HOSTNAME**
            comment (str): comment为日志的注释信息Comment log_dir suffix appended to the default
              ``log_dir``. If ``log_dir`` is assigned, this argument has no effect.
            purge_step (int):当日志记录在步骤:math: ' T+X '崩溃并在步骤:math: ' T '重新启动时，global_step大于或等于:math: ' T '的任何事件从TensorBoard中清除和隐藏。请注意，崩溃和恢复的实验应该具有相同的“log_dir”。
              When logging crashes at step :math:`T+X` and restarts at step :math:`T`,
              any events whose global_step larger or equal to :math:`T` will be
              purged and hidden from TensorBoard.
              Note that crashed and resumed experiments should have the same ``log_dir``.
            max_queue (int):summary队列的最大长度，默认值为10。 Size of the queue for pending even  Default is ten items.
            flush_secs (int): 每隔多少秒将数据刷新到磁盘上，默认值为120秒。to flush the pending events and summaries to disk. Default is every two minutes.
            filename_suffix (str): 输出日志文件的后缀名，默认值为“events.out.tfevents”，可以自定义设置 More details on filename construction in
              tensorboard.summary.writer.event_file_writer.EventFileWriter.
          """

add_scalar() function

Parameter Description:

      Args:参数说明
            tag (str): Data identifier                                 #相当于title
            scalar_value (float or string/blobname): Value to save    #相当于y轴
            global_step (int): Global step value to record           #相当于x轴
            walltime (float): Optional override default walltime (time.time())
              with seconds after epoch of event
            new_style (boolean): Whether to use new style (tensor field) or old
              style (simple_value field). New style could lead to faster data loading.  #指定图片数据的格式

Example of use:

from torch.utils.tensorboard import SummaryWriter
writer=SummaryWriter("logs")
 for i in range(100):
     writer.add_scalar("y=x",i,i)
writer.close()

The operation of SummaryWriter will generate the event file of tensorbord.

How to use the time file.

Result display:

when modified to y=2X

There is a problem: There is a problem with the image

因为第二个事件文件的展示是在第一个上面添加的
解决办法：

1. First delete the two events files and run again
2. writer=SummaryWriter("logs") creates subfolder

add_image() function

Parameter Description:

   def add_image(
        self, tag, img_tensor, global_step=None, walltime=None, dataformats="CHW"
    ):
        """Add image data to summary.
        Args:
            tag (str): Data identifier    #相当于title
            img_tensor (torch.Tensor, numpy.ndarray, or string/blobname): Image data   图像数据（有格式要求）
            global_step (int): Global step value to record      
            walltime (float): Optional override default walltime (time.time())
              seconds after epoch of event
            dataformats (str): Image data format specification of the form  CHW, HWC, HW, WH, etc.

The data format used to read images using openCV is complex.

How to convert it into the format it allows?

Image can be seen

Modify

#coding=gbk
from torch.utils.tensorboard import SummaryWriter
import cv2
## 创建这个类的实例
writer=SummaryWriter("logs")
rgb_dir="D:\\AdeepLearningTest\\Code\\NYUDepthv2\\RGB\\1.jpg"
img=cv2.imread(rgb_dir);
print(img.shape)
#Default is :math:`(3, H, W)所以要指定格式为HWC
writer.add_image("image1",img,2,dataformats='HWC')#这个1指的是步骤一，改成步骤2
writer.close()

The yellow line can be switched by sliding left or right