just watched mask rcnn yesterday
Write down your impressions
The mask rcnn is based on the faster rcnn and adds a parallel branch for instance segmentation.
And the previous RoIPooling is replaced by RoIAlign. The advantage of using RoIAlign is that the use of bilinear interpolation can retain the information before the operation more specifically, which is conducive to instance segmentation.
The paper on mask rcnn is relatively simple. In instance segmentation, FCN is used, which is about as much as I can think of now.
The key RoIAlign of mask rcnn is to change the RoIPooling interpolation method from the nearest neighbor interpolation (INTER_NEAREST) method to bilinear interpolation.
Here's a refresher on RoI, RoIPooling
RoI is the region of interest ("box") on the Region of Interests feature map
The process of RoIPooling is to map boxes of different sizes into rectangular boxes of fixed size (w*h).
The specific operation is: first map the coordinates in the RoI to the feature map, and after obtaining the coordinates on the feature map, use Pooling to get the output. During the Pooling process, it is necessary to calculate the range occupied by the pooling result corresponding to the feature map, and then Take the max or average operation in that range. Using RoIPooling will introduce misalignments between RoI and extracted features.
So use RoIAlign to solve this problem.
The paper also compares with RoIWarp ( J. Dai, K. He, and J. Sun. Instance-aware semantic segmentation via multi-task network cascades. In CVPR , 2016. 2 , 3 , 4 , 5 , 6 ), The experimental results show that the effect of RoIAlign is better than that of RoIWarp.
Tried two different structures of faster rcnn, ResNet C4 and FPN structure.
As can be seen from the figure, ResNet C4 shares the RoI layer, while FPN separates the two tasks for RoI.
mask Loss is only defined on positive RoIs.
In this article, the RPN and mask branches do not share features during training; for the previous feature extraction, RPN and mask share the basic network structure.
