CVPR2020 is a highly evaluated matting algorithm, which is very suitable for static background related fields. It can be used for high-fine-grained foreground extraction and background replacement (which can be used for small sample data enhancement). At present, github open source has more than three thousand stars.
Paper address: https://arxiv.org/abs/2004.00626
Code: https://github.com/senguptaumd/Background-Matting
Before the principle of the algorithm, it needs to be introduced. The physical model of the cutout is based on
I = αF + (1 − α)B, it is easy to understand, I is the original image, F is the foreground, B is the background, and alpha can be understood as the foreground heatmap
The main idea of the algorithm consists of two parts, as shown in the figure above:
The first part, the foreground matting network: the input includes four parts, the original image I, the background B, the segmentation result S, the front and rear frame grayscale image M, after encoding, the B, S, and M sampling results are respectively concat with the I sampling results, After a step of convolution, all the information is forwarded and upsampled to get the output, alpha and foreground
After experimenting, the author found that the matting effect of the network trained solely using the Adobe dataset is not ideal after changing the scene, so the second step is cascaded, and the GAN part is cascaded. The part of the GAN initializes the new matting network Greal. You need to use the network Gadobe trained in the first step, use Gadobe as a teacher network and combine the loss of the discriminator (Discriminator) to train Greal, and finally, update the discriminator parameters every five iterations