1. Description

I think, anyone can try to remove the background from the image. Of course, there are tons of software or tools available to do this, but some of them can be expensive. However, I know people who use Window Paint 3D Magic Selection or PowerPoint Background Removal to remove the background.

If you are a beginner in computer vision, this might be for you. Let's get started right away! Oh, without using much code...

2. Introduction of GrabCut

GrabCut is a computer vision algorithm for extracting objects from images. Because it doesn't always work the first time, Interactive GrabCut allows the user to indicate how to improve the output. On the web, there are several GrabCut implementations; some only run in Python (no web interface), while others are not interactive.

2.1 What is the process of using GrabCut?

1. The user enters the rectangle. Everything outside this rectangle will be considered the background. Everything inside the rectangle is a mystery.

2. Algorithms mark pixels (or hard marks) in the foreground and background

3. The foreground and background are then modeled using a Gaussian Mixture Model (GMM).

4. GMM learns and creates new pixel distributions based on the data we provide. In other words, unknown pixels are labeled as possible foreground or likely background (like clustering) based on their color statistical relationship to other hard-labeled pixels.

5. This pixel distribution is used to create the graphic. Pixels are nodes in a graph. Two new nodes have been added: a "Source" node and a "Sink" node. Each foreground pixel is linked to a source node.

Credit researchgate.net

2.2 How to use the GrabCut package

2.2.1 Step 1: Import the necessary packages

First, we must import the necessary packages and override our matplotlib functions.

code segment:

import cv2
import dlib
import sys
import numpy as np
from tkinter import filedialog
from matplotlib import pyplot as pltdef imshow(title = "Image", image = None, size = 10):
    w, h = image.shape[0], image.shape[1]
    aspect_ratio = w/h
    plt.figure(figsize=(size * aspect_ratio,size))
    plt.imshow(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))
    plt.title(title)
    plt.show()

2.2.2 Step 2: Choose the ROI function

We can use this OpenCV method to carefully manually select the region of interest we need from the image.

original photo

Courtesy of Eskipaper.com

img = r"C:\Users\jinzh\Desktop\Project\Python\python-opencv\lovely-girl-background-1.jpg"

image = cv2.imread(img)
copy = image.copy()
# Create a mask (of zeros uint8 datatype) that is the same size (width, height) as our original image 
mask = np.zeros(image.shape[:2], np.uint8)bgdModel = np.zeros((1,65), np.float64)
fgdModel = np.zeros((1,65), np.float64)x, y , w, h = cv2.selectROI("select the area", image)start = (x, y)
end = (x + w, y + h)rect = (x, y , w, h)
cv2.rectangle(copy, start, end, (0,0,255), 3)
imshow("Input Image", copy)

output

Generated from Jupyter Notebook (author)

2.2.3 Part 3 Grab and Cut Algorithm Algorithm:

Crawl parameters

1. img — input image

2. Mask — This is a mask image that specifies which parts are background, foreground, or possible background/foreground, etc. Flags cv.GC_BGD, cv.GC_FGD, cv.GC_PR_BGD, cv.GC_PR_FGD, or just pass 0, 1, 2, 3 to image to do this.

3. rect – in format, it is the coordinates of the rectangle containing the foreground object (x, y, w, h)

4. bdgModel and fgdModel — used internally by the algorithm. You just need to create two zero arrays of type np.float64, each of size np.float64(1, 65).

5. iterCount — the number of iterations the algorithm should run.

6. mode — this should be cv.GC_INIT_WITH_RECT, cv.GC_INIT_WITH_MASK, or a mix, depending on whether we're drawing rectangles or adding touches.

cv2.grabCut(image, mask, rect, bgdModel, fgdModel, 100, cv2.GC_INIT_WITH_RECT)
mask2 = np.where((mask==2)|(mask==0),0,1).astype('uint8')
image = image * mask2[:,:,np.newaxis]imshow("Mask", mask * 80)
imshow("Mask2", mask2 * 255)
imshow("Image", image)

explain

1. Run the algorithm for 5 iterations. Since we are using rectangles, the mode should be cv.GC_INIT_WITH_RECT.

2. The mask image is modified by Grabcut.

3. As above, the pixels in the new mask image will be marked with four flags indicating background/foreground.

4. So we change the mask so that all 0 and 2 pixels are set to 0 (background) and all 1 and 3 pixels are set to 1. (i.e. foreground pixels).

5. Our last mask is now complete. To get a segmented image, just multiply it by the input image.