[PIDNet](GitHub - XuJiacong/PIDNet: This is the official repository for our recent work: PIDNet)
It is a new open source real-time semantic segmentation algorithm in 22 years, which has the same good performance as DDRNet
The network structure is as follows:
The network is divided into three branches, as shown in the figure above, the overall structure is similar to DDRNet, including the source code is also similar to DDRNet.
One more branch here adds information about the segmentation edge.
Here, the tensorrtx method is used to convert the network layer by layer into a TensorRT model. The details of the project will not be described, see: https://github.com/midasklr/PIDNet.TensorRT/blob/main/READMECH.md
Note some implementation details:
-
The cityscape data set training of the Pytorch source code is: input data 2048x1024, then preprocess, randomly crop out 1024x1024 training, the input is 2048x1024 during the test, here the TensorRT model uses the input data to resize to 1024x1024 and then reasoning, the actual measurement loss is not large, the main reason is that During the accuracy correction of the tensorrt and pytorch models, it was found that this input loss was the smallest (very small).
-
During training, read pictures => 1024x512 => randomly crop 512x512 for training. Compared with the original operation training, the speed is increased by 3+ times, and the training time is reduced from one week to 2 days. The cost is that the mIOU is reduced by about 1%.
-
The source code uses DP mode to support multi-card training, and the test found that it is not as fast as single-card training.
-
PAPPM module modification. The receptive field of pooling in the original PAPPM is used to adapt to the 2048x1024 input, and becomes (1, 512, 32, 64) when the PAPPM module is input, corresponding to kernel=5, 9, 17, and global. When the input is changed to 1024x512, spp input (1,512,16,32), how to adjust the pooling kernel.
-
Note that the convolution in scale_process in the PAPPM module is group=4 group convolution!
TensorRT源码：GitHub - midasklr/PIDNet.TensorRT: TensorRT project of PIDNet