[OpenCV-Python] Tutorial: 3-15 Watershed Image Segmentation

OpenCV Python watershed image segmentation

【Target】

• Learning to use watershed methods for marker-based image segmentation
• cv2.watershed()

【theory】

Any grayscale image can be considered a topographic surface, where high intensities represent peaks and hills, and low intensities represent valleys. You start by filling each isolated valley (local minima) with a different color of water (label). As the water rises, water from different valleys, obviously of different colors, will start to merge, depending on the nearby peaks (gradient). To avoid this, you need to build barriers where the water joins, you keep filling water and building barriers until all the peaks are submerged, then, the barriers you create give the segmentation result. This is the "philosophy" after the watershed parting, you can visit the webpage on the watershed to watch the animation, as follows:

However, due to noise or any other irregularities in the image, this approach will produce over-segmented results, so, OpenCV implements a marker-based watershed algorithm, where you formulate which are all the valley points to be merged, Which ones are not, this is a kind of interactive image segmentation, what we do is assign different labels to the objects that we know, mark the areas that we know are foreground or objects with a color. Mark the areas we identify as background or non-objects with another color. Finally, mark the area we are not sure about with 0, this is our mark, then apply the watershed algorithm, then, our mark will be updated with the label we gave and the boundary of the object will be -1.

• Segmentation flow chart

1. Find markers and segmentation criteria (criteria or functions are often used to separate regions, often contrast or gradient, but not necessary.
2. Implements the watershed algorithm for marker control.

【Code】

Segment sticky objects using distance transform and watershed.

import numpy as np 
import cv2 
from matplotlib import pyplot as plt

# 读入图像
img = cv2.imread('assets/water_coins.jpg')
gray = cv2.imread('assets/water_coins.jpg', 0)

# 阈值化
ret, thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU )

# 开运算（先腐蚀后膨胀），去噪声
kernel = np.ones((3, 3), np.uint8)
opening = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel, iterations = 2)

# 背景区域
sure_bg = cv2.dilate(opening, kernel, iterations = 3)

# 通过距离变换，然后阈值化找前景
dist_transform = cv2.distanceTransform(opening, cv2.DIST_L2, 5)
# 这个地方的阈值（0.5 * dist_transform.max()）调节很重要，直接关系到后面的分割效果
ret, sure_fg = cv2.threshold(dist_transform, 0.5 * dist_transform.max(), 255, cv2.THRESH_BINARY)

# 计算未知区域
sure_fg = np.uint8(sure_fg)
unknown = cv2.subtract(sure_bg, sure_fg)

# 标记连通区域
ret, markers = cv2.connectedComponents(sure_fg)

# 所有的标记+1，背景改为1，而不是0
markers = markers + 1

# # 标记不确定区域为0
markers[unknown==255] = 0

# 用分水岭方法找到标记，如果标记为 -1，则该位置设置蓝色，即边缘颜色
markers = cv2.watershed(img, markers)
img[markers == -1] = [255, 0, 0]

# 显示分割的图像
image2 = np.uint8(img)
cv2.imshow("img2", image2)

# 显示各阶段的图像
plt.subplot(231), plt.imshow(gray, 'gray'), plt.title(
    'Original'), plt.xticks([]), plt.yticks([])
plt.subplot(232), plt.imshow(sure_bg, 'gray'), plt.title(
    'sure_bg'), plt.xticks([]), plt.yticks([])
plt.subplot(233), plt.imshow(sure_fg, 'gray'), plt.title(
    'sure_fg'), plt.xticks([]), plt.yticks([])
plt.subplot(234), plt.imshow(dist_transform), plt.title(
    'dist_transform'), plt.xticks([]), plt.yticks([])
plt.subplot(235), plt.imshow(markers), plt.title(
    'markers'), plt.xticks([]), plt.yticks([])
plt.subplot(236), plt.imshow(img), plt.title(
    'img'), plt.xticks([]), plt.yticks([])
plt.show()

cv2.waitKey(0)
cv2.destroyAllWindows()

If not markers = markers + 1, the result is:

If not markers[unknown==255] = 0, the result is:

Therefore, the generation and selection of the seed point 0 value is very important, which directly affects the final result.

【interface】

• distanceTransform

cv.distanceTransform(	src, distanceType, maskSize[, dst[, dstType]]	) ->	dst
cv.distanceTransformWithLabels(	src, distanceType, maskSize[, dst[, labels[, labelType]]]	) ->	dst, labels

Computes the exact or approximate distance from each pixel in the image to the nearest 0 pixel . If a zero-valued image, then the distance is of course 0. When maskSize == DIST_MASK_PRECISEand distanceType == DIST_L2, run the algorithm [73] , the function has been optimized for parallelization with TBB. In other cases, the algorithm [29] is used . This means that the path to find the nearest zero pixel can be horizontal, vertical, diagonal, Knight's Move (knight movement?), and the overall distance is calculated through a series of basic distances. Horizontal and vertical aare represented by , diagonal bis represented by , knight movement is crepresented by

DIST_L1: a = 1, b=2

DIST_L2:
3 x 3: a=0.955, b=1.3693
5 x 5: a=1, b=1.4, c=2.1969
DIST_C: a = 1, b = 1
Usually, for fast rough distance estimation, use DIST_L2 3x3 mask, If accurate, use DIST_L2 5x5 mask. Regardless, all of these exact or approximate distances are linear functions of the number of pixels.

• src : 8-bit single-channel binary image
• dst : The image of the distance calculation result, which can be an 8-bit or 32-bit floating-point single-channel image, and the image size is consistent with the source image;
• labels : Output 2D labels (also Voronoi diagram - Thiessen polygon diagram), 32-bit single channel.
• distanceType : distance type
• maskSize : Mask of distance transformation
• labelType : label type see DistanceTransformLabelTypes.

• distanceType distance type

• DistanceTransformMasks distance transformation mask

• DistanceTransformLabelTypes distance transform label type

• connectedComponents

cv.connectedComponents(	image[, labels[, connectivity[, ltype]]]	) ->	retval, labels

cv.connectedComponentsWithAlgorithm(	image, connectivity, ltype, ccltype[, labels]	) ->	retval, labels

Calculate connected domain labels in binary images
Support Bolelli (Spaghetti) [26] , Grana (BBDT) [97] and Wu's (SAUF) [278] algorithms;

• image : 8-bit single-channel image
• labels : the labels of the target image
• connectivity : 4 neighborhoods or 8 neighborhoods
• ltype : The output image label type supports CV_32S, CV_16U
• ccltype : connected domain algorithm type ConnectedComponentsAlgorithmsTypes

• ConnectedComponentsAlgorithmsTypes

• watershed

cv2.watershed(	image, markers	) ->	markers

Performs segmentation based on labeled images, using the watershed algorithm. [171] .
Before passing an image to a function, the desired region must be roughly outlined in image markers with positive (>0) indices. Thus, each region is represented as one or more connected components with pixel values ​​1, 2, 3, etc. Such markers can be retrieved from a binary mask using findContours and drawContours (see watershed.cpp for a demo). Markers are "seeds" for future image regions. All other pixels in the marker (whose relationship to the contour area is unknown and should be defined by the algorithm) should be set to 0. In the function output, each pixel in the marker is set to the value of the "seed" component, or -1 at the boundaries between regions.

• image : 8-bit 3-channel image
• markers : 32-bit single-channel marker image for input and output;

【reference】

1. OpenCV Official Documentation
2. CMM page on Watershed Transformation
3. Pedro Felzenszwalb and Daniel Huttenlocher. Distance transforms of sampled functions. Technical report, Cornell University, 2004.
4. Gunilla Borgefors. Distance transformations in digital images. Computer vision, graphics, and image processing, 34(3):344–371, 1986.
5. Federico Bolelli, Stefano Allegretti, Lorenzo Baraldi, and Costantino Grana. Spaghetti Labeling: Directed Acyclic Graphs for Block-Based Connected Components Labeling. IEEE Transactions on Image Processing, 29(1):1999–2012, 2019.
6. Costantino Grana, Daniele Borghesani, and Rita Cucchiara. Optimized Block-Based Connected Components Labeling With Decision Trees. IEEE Transactions on Image Processing, 19(6):1596–1609, 2010.
7. Kesheng Wu, Ekow Otoo, and Kenji Suzuki. Optimizing two-pass connected-component labeling algorithms. Pattern Analysis and Applications, 12(2):117–135, Jun 2009.
8. Fernand Meyer. Color image segmentation. In Image Processing and its Applications, 1992., International Conference on, pages 303–306. IET, 1992.