Python image processing - image segmentation and color-based image segmentation of classic cases

In the preface,
the author introduces the basic knowledge of image processing in the first part. The second part introduces image computing and image enhancement. Next, in the third part, we will explain in detail the classic cases of image segmentation and image processing. This part belongs to high-level image processing. Knowledge can further deepen our understanding and practical ability. Image segmentation is the technology and process of dividing an image into several regions with unique properties and extracting objects of interest. It is a key step in image processing and image analysis. It is mainly divided into threshold-based segmentation methods, region-based segmentation methods, edge-based segmentation methods, and specific theory-based segmentation methods. This article will explain in detail the color-based image segmentation method.

1. Color-based image segmentation

Color segmentation is an important technology in image processing. It can separate different colors in an image for application scenarios such as target detection and image recognition.

The algorithm principle of color segmentation is relatively simple, and it is usually realized based on the transformation of color space. Commonly used color spaces include RGB, HSV, LAB, etc., among which the HSV color space is more suitable for color segmentation tasks. The HSV color space divides colors into three channels: Hue, Saturation, and Value.

• Hue: Indicates the kind or type of color, such as red, blue, green, etc.
• Saturation: Indicates the purity of the color, the more saturated the color is, the more vivid it is, and vice versa.
• Brightness: Refers to the lightness and darkness of a color. In color, the higher the brightness, the brighter the color; the lower the brightness, the darker the color.

The basic idea of ​​color segmentation is to segment pixels of different colors by thresholding. Specifically, we can convert the image from the RGB color space to the HSV color space, and then select an appropriate threshold (usually the maximum and minimum values ​​of hue, saturation, and brightness) to divide the image into black and white parts.

In opencv hsv color space

• The maximum value of hue (Hue) is 179 degrees, and the minimum value is 0 degrees, which represents the position of the color on the color wheel.

• The maximum value of Saturation is 255, and the minimum value is 0, representing the purity and grayscale of the color.

• The maximum value of Brightness is 255, and the minimum value is 0, which represents the lightness and darkness of the color.

Implementation steps
Based on the above algorithm principles, we can perform the following steps to achieve color segmentation:

1. Convert an image from RGB color space to HSV color space.
2. Calculate the maximum and minimum values ​​of Hue, Saturation, Brightness respectively
3. Set the threshold to split the image into black and white parts.
4. Use morphological operations to remove noise and get the final segmentation result.

This paper uses the cv2.inRange() function in OpenCV to perform color-based segmentation of the target in the image.

mask = cv2.inRange(src, lowerb, upperb[, dst])

— The src parameter is the input image, which can be single-channel or multi-channel, and the data type is np.uint8
— mask is the output image, which is a single-channel image (binary image), and the data type is the same as the input image
— lowerb is the specified lower limit , of type Scalar, which can be a single value or a tuple or list of multiple values
—upperb is the specified upper bound, of type Scalar, which can be a single value or a tuple or list of multiple values

Note:
The mask represents the position information of the pixels matching the specified color range, that is, the corresponding pixel value of the pixels within the specified color range in the binary image is 255, and the pixels not in the specified color range are in the binary image The corresponding pixel value in is 0. Therefore, by using this mask, we can selectively operate on the image and only process pixels within the specified color range without affecting other pixels

The following is a simple color segmentation code based on the above algorithm principle (the mask itself gives a given range):

import cv2
import numpy as np

# 读入图片
img = cv2.imread( "C:/Users/Administrator/Desktop/flower.png")

# 转换颜色空间
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)

# 定义绿色范围
lower_green = np.array([40, 50, 50])
upper_green = np.array([90, 255, 255])

# 定义黄色范围
lower_yellow = np.array([15, 50, 50])
upper_yellow = np.array([40, 255, 255])

# 根据颜色范围创建掩码
mask_green = cv2.inRange(hsv, lower_green, upper_green)
mask_yellow = cv2.inRange(hsv, lower_yellow, upper_yellow)

# 合并掩码
mask = cv2.bitwise_or(mask_green, mask_yellow)

# 应用掩码
result = cv2.bitwise_and(img, img, mask=mask)

# 显示结果
cv2.imshow('result', result)
cv2.waitKey(0)
cv2.destroyAllWindows()

The running results are shown in Figure 1-1. The sunflowers are well segmented in the figure, but the grass is also segmented.

The following are the HSV ranges of some common colors:

Red: (0, 70, 50) - (10, 255, 255) or (170, 70, 50) - (179, 255, 255)

Orange: (10, 70, 50) - (25, 255, 255)

Yellow: (25, 70, 50) - (35, 255, 255)

Green: (35, 70, 50) - (85, 255, 255)

Cyan: (85, 70, 50) - (100, 255, 255)

Blue: (100, 70, 50) - (130, 255, 255)

Purple: (130, 70, 50) - (170, 255, 255)

White: (0, 0, 221) - (180, 30, 255)

Black: (0, 0, 0) - (180, 255, 30)

2. Determine the color value range by observing the histogram

When the color target to be segmented has no major interference in the image, the value range of the color target can be determined by observing the histogram of the hue (Hue) component, as the boundary standard for our segmentation.

The code example is as follows:

import cv2
import numpy as np
from matplotlib import pyplot as plt

#通过OpenCV读取图片信息
img = cv2.imread("C:/Users/Administrator/Desktop/scenery.jpg")
# BGR图转为HSV
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
# 提取hsv中H通道数据
h = hsv[:, :, 0].ravel()
# 直方图显示
plt.hist(h, 180, [0, 180])
plt.show()

Get the histogram, as shown in Figure 2-1:

According to the HSV range of common colors mentioned in the appeal, a large amount of data appears in the 30 to 70 part of the histogram, so it can be determined that the upper and lower limits of the green H component are 30-70, The limit of SV can be roughly set according to the range of HSV proposed in the appeal, and it can be adjusted and determined when extracting the mask

The mask mask is generated below through the green boundary of HSV. code show as below:

import cv2
import numpy as np
from matplotlib import pyplot as plt

#通过OpenCV读取图片信息
img = cv2.imread("C:/Users/Administrator/Desktop/scenery.jpg")
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)

# HSV 的下界限
lower_red = np.array([35,70,50])
# HSV 的上界限
upper_red = np.array([70,255,255])

# 通过上下限提取范围内的掩模mask
mask = cv2.inRange(hsv, lower_red, upper_red)

img =  cv2.resize(img,(500,500))
mask = cv2.resize(mask,(500,500))

cv2.imshow("img", img)
cv2.imshow("mask", mask)
cv2.waitKey(0)
cv2.destroyAllWindows()

The running result is shown in Figure 2-2

It can be found that the general outline of the grass has been extracted, but there are many black spots on the field. This is because most of the black spots are in the dark, and we can filter them out by reducing the lower limit of the V value. Here, the lower limit of the V value is determined by The original 50 is adjusted to 15

The effect is as follows:

At this time, some black spots cannot be removed by V adjustment, we can use expansion, corrosion and other means to eliminate independent white spots or black spots.

The code to eliminate black spots is as follows:

import cv2
import numpy as np
from matplotlib import pyplot as plt

#通过OpenCV读取图片信息
img = cv2.imread("C:/Users/Administrator/Desktop/scenery.jpg")
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)

# HSV 的下界限
lower_green = np.array([35,70,15])
# HSV 的上界限
upper_green = np.array([70,255,255])

# 通过上下限提取范围内的掩模mask
mask = cv2.inRange(hsv, lower_green, upper_green)

# 定义结构元素
kernel = cv2.getStructuringElement(cv2.MORPH_RECT, (5, 35))
# 图像闭运算
mask = cv2.morphologyEx(mask, cv2.MORPH_CLOSE, kernel, iterations=2)

img =  cv2.resize(img,(500,500))
mask = cv2.resize(mask,(500,500))

cv2.imshow("img", img)
cv2.imshow("mask", mask)
cv2.waitKey(0)
cv2.destroyAllWindows()

The effect is shown in Figure 2-3.

It can be seen that the black points are eliminated, and then we use cv2.bitwise_and() to bitwise and the original image and the mask to extract the grass picture:

color replacement

Through the above operations, we can already accurately segment the color we specified, and then we can change its color on a second basis, such as changing it to yellow. This also needs to be operated in the HSV color space, but this time There is no need to perform a closing operation on the mask.

Use the createTrackbar function to adjust hsv here

cv2.createTrackbar(trackbarName, windowName, value, count, onChange)

• trackbarName: the name of the draggable bar, string type.
• windowName: The name of the window where the draggable bar is located, string type.
• value: the default value of the draggable bar, integer.
• count: the maximum value of the draggable bar, integer.
• onChange: the callback function called when the value of the draggable bar changes, function type.

code show as below:

import cv2
import numpy as np

def nothing(x):
    pass
#通过OpenCV读取图片信息
img = cv2.imread("C:/Users/Administrator/Desktop/scenery.jpg")

img = cv2.resize(img,(500,500))
cv2.namedWindow('img',cv2.WINDOW_NORMAL)
cv2.imshow("img", img)

# HSV 的下界限
lower_green = np.array([35,70,15])
# HSV 的上界限
upper_green = np.array([70,255,255])

cv2.namedWindow('img2',cv2.WINDOW_NORMAL)

cv2.createTrackbar('H','img2',140,180,nothing)
cv2.createTrackbar('S','img2',100,180,nothing)
cv2.createTrackbar('V','img2',117,180,nothing)
rows,cols,channels = img.shape

while(1):
    hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)
    mask = cv2.inRange(hsv, lower_green, upper_green)
    #将制定像素点的数据设置为0, 要注意的是这三个参数对应的值是Blue, Green, Red。
    h = cv2.getTrackbarPos('H', 'img2')
    s = cv2.getTrackbarPos('S', 'img2')
    v = cv2.getTrackbarPos('V', 'img2')
    for r in range(rows):
        for c in range(cols):
            if mask[r, c] == 255:
                hsv.itemset((r, c, 0), hsv.item(r, c, 0) -h)
                hsv.itemset((r, c, 1), hsv.item(r, c, 1) +90-s)
                hsv.itemset((r, c, 2), hsv.item(r, c, 2) +90-v)
    img2 = cv2.cvtColor(hsv, cv2.COLOR_HSV2BGR)
    #将图像进行输出，使用show()也是可以显示的。
    img = cv2.resize(img, (500, 500))
    cv2.imshow("img2", img2)
    k = cv2.waitKey(1)&0xFF
    if k == 27: #esc exit
        break

#cv2.waitKey(0)
cv2.destroyAllWindows()

3. Summary

This blog introduces how to use Python and OpenCV for color segmentation, which uses the HSV color space and threshold segmentation algorithm. The steps to achieve color segmentation include: reading image files, converting color spaces, calculating thresholds, segmenting images, removing noise, etc. Through the code of the example, we can have a deeper understanding of the implementation process of color segmentation.