Stable Diffusion's background removal tool "Remove background" is an extremely important tool in image processing. Compared with traditional matting tools, this tool is more intelligent, which greatly improves the efficiency and accuracy of the matting process. This tool can adapt to images of different scales and complexities, and adapt to different matting tasks by adjusting parameters.
Article Directory
Plug-in Introduction
The plug-in interface is Chinese, which is convenient and easy to use. To install this plugin, you can do it through the "Extensions" tab. I have already installed it here, so it shows installed.
Plug-in URL download address.
GitHub - AUTOMATIC1111/stable-diffusion-webui-rembg
After installation, this function will be added at the bottom of your Extras additional function tab.
When using Remove backgroundthe tool, you can also choose not to zoom the picture to be cut, just select 1 for the zoom ratio, so that you can purely cut out the picture.
Matting Algorithm Selection
The default scaling tool comes with u2net, u2netp, u2net_human_seg, u2net_cloth_seg, silueta algorithms. The following is a brief introduction to these algorithms and their corresponding effects:
- u2net : U2-Net is a deep learning model that is mainly used for salient object detection, that is, identifying the object that most attracts visual attention in an image. The name of this model comes from its unique nested U-shaped structure, which has the characteristics from shallow to deep, and can obtain multi-scale and multi-level context information.
- u2netp : U2-NetP is a lightweight version of the U2-Net model. It is basically the same as U2-Net in terms of structure and function, but the model is smaller and the calculation efficiency is higher, which is suitable for environments or applications with limited computing resources.
- u2net_human_seg : U2-Net Human Seg is a special version of the U2-Net model, optimized and used for human segmentation tasks. It has a good effect on the outline and detail processing of portraits.
- u2net_sloth_seg : U2-Net Cloth Seg may be a specialized version of the U2-Net model, which is specially used to process and extract image information of clothing. Note that this is speculation, as no definition of this name has been found in academic literature or open source projects so far.
- silueta : The name means "silhouette" in Spanish, probably some kind of algorithm for processing image contours. Again, no definition of this name has been found in academic literature or open source projects so far.
- isnet-general-use : This may be the name of another deep learning model, but no specific definition or description of this name was found. Probably it is a model for general image processing tasks.
- isnet-anime : This is probably a specialized version of the ISNet model designed specifically for processing anime-style images. However, again, no definition of this name has been found in the academic literature or in open source projects.
Depending on the accuracy required, you can choose among these algorithms.
| Cutout method | Effect demo 1 | Effect demo 2 |
|---|---|---|
| original image |  |
 |
| u2net |  |
 |
| u2netp |  |
 |
| u2net_human_seg |  |
 |
| u2net_sloth_seg |  |
 |
| silhouette |  |
 |
| isnet-general-use |  |
 |
| isnet-anime |  |
 |
extra parameters
In addition to the above algorithm selection, there are two options of and Remove backgroundin the tool . Alpha matting is a technique for separating foreground objects from the background, where the separation is achieved by computing an alpha value between the foreground object and the background.Return maskAlpha matting
- FOREGROUND_THRESHOLD Foreground Threshold : This parameter is used to define which parts should be considered as the foreground of the image. Alpha values generally represent the transparency of a pixel, ranging from 0 (fully transparent) to 255 (fully opaque). When the Alpha value is higher than the foreground threshold, those pixels will be identified as foreground. The setting of this value needs to take into account the specific content and expected effect of the image.
- Background_THRESHOLD background threshold : This parameter is similar to the foreground threshold, but it is used to define which parts should be considered as the background of the image. When the Alpha value is lower than the background threshold, those pixels will be identified as the background. Similarly, the setting of this value also needs to take into account the specific content and expected effect of the image.
- ERODE_SIZE Alpha matting erosion size : Erosion is an image processing technique, mainly used to remove noise in the image, separate two connected objects, etc. The size of the erosion operation is the size of the area on the image where the erosion operation is applied, usually a square area. The larger the corrosion size, the more obvious the corrosion effect, but too large corrosion size may lose some important image information.
The use of specific parameters can be adjusted according to personal preferences and needs.
| Recommended parameters | value |
|---|---|
| -Erode size | 10 |
| Foreground threshold | 220 |
| Background threshold | 40-100 |
Some parameters are listed below to compare the test results:
| method | renderings |
|---|---|
| original image |  |
| mask |  |
| F-20 B-0 |  |
| F-100 B-0 |  |
| F-200 B-50 |  |
| F-220 B-100 |  |
| After the parameters are fine-tuned, the black border of the character cutout will disappear, and the use effect will be better. |