Article directory
foreword
本人之前相对视频做一个抠图处理，奈何手工逐帧抠肯定不行，本来想在网上找找有没有好用且免费的抠图工具，看到有非常多人推荐一款在线抠图的网站，叫什么无绿幕抠图。我就抱着好奇点开试了一试，结果效果还是很差，而且只是生成了gif动图。综合观察，奈何要么收费，要么就是效果很差。
Therefore, I also collected a good matting algorithm at present, and saw a paper "Robust High-Resolution Video Matting with Temporal Guidance". So I wanted to learn from it to make a matting tool that can be used directly by people. But the effect is not very good, and there are still some bugs in the tool, but let’s use it for now, and those who need it can try it.
提示：工具只是个半成品，打开、关闭还有处理过程中还没完善。
1. Video and Picture processing effect
Example: In scenes where the portrait background is relatively monotonous, the effect is still good. However, the effect is poor where the background and the portrait color are similar, which is also the main problem that most algorithms currently solve.
1. Image cutout effect
2. Video cutout effect
Summary:
Compared with free cutout tools, the effect of video cutout is good, but there are still some blurred pictures that are not handled very well. In terms of picture cutout effect, some scenes may have poor effect.
Second, the method used
The author's original code is a Unet-like structure model trained on the pytorch environment. The main categories are encoding, decoding, and upsampling output. I won’t go into details here. You can refer to other people’s analysis blogs .
The data sets used for training include: portrait semantic segmentation data set, matting map data set. The losses used include: alpha channel L1 loss and Laplacian pyramid loss, timing correlation loss, L1 loss of fg part and timing correlation loss. Converting the model weight file to an onnx model originally wanted to be quantized, so the processing time may be a little longer now, so you can consider accelerating the prediction processing speed in the future, adding half-precision selection, model pruning and quantization processing.
3. Use steps
Directly open Matting tool by CS.exe in the root directory.
1.Data selection
Input Dir : Select the video and picture to be processed;
Output Dir : Select the save location folder;
Model weight : Model weight, select model.onnx under the weights folder;
2.Advance settings
Show Result : The processing process can be displayed in real time (but due to the problem of the display control, it pops up), but since the cv2 display will affect the processing speed, you can consider removing the tick; mode: choose whether to process video
or pictures;
background color : choose to cut The background color of the graph, the default is green screen;
3.Start Run
Click the Start Run button to start processing, and you can view the current processing progress through the command line window (be careful not to close this window during processing). If you don't want to deal with it, you can click the Close Run button to end the process.
The final output is output(+audio).mp4 video or out.jpg image.
The tool can be obtained from the network disk. Currently, there are many bugs and the effect is limited. It is only for entertainment.
··· Baidu cloud disk ··· extraction code: 1122
Summarize
The training effect is limited, and the processing speed is limited by the equipment configuration. In the future, a pruning and quantization process will be considered.
Update: V1.3
is updated again. Some friends reported that they need to process pictures in batches . I modified the original function button and changed the input to Input Images (select a folder of multiple pictures) and Input Video (select a single video file. ). This enables batch processing of images.
Update [b] download address is here:
··· Baidu cloud disk ···
Extraction code: 1210.
Download and install the software, there is a little Easter egg inside, if you like it, you can give it a reward, thank you!
The video frame insertion software is still in the process of expedited production. Happy Labor Day!