1 Introduction
Welcome back, fellow image manipulation enthusiasts! In this article, we will directly enter a new field of traditional image analysis - image segmentation, which refers to the process of dividing an image into several regions with similar properties. From a mathematical point of view, image segmentation is the process of dividing an image into mutually disjoint regions.
Without further ado, let's get started!
2. Threshold-based segmentation
Firstly, the threshold-based and Otsu-based segmentation methods are introduced. This is the first step in our segmentation experiment. We will see how simple and effective methods like this can divide the image into foreground and background according to the intensity value of the image pixels.
But how do we scientifically decide the threshold for segmentation? This is where Otsu's method comes in handy. Briefly, this ingenious approach makes it an excellent tool for adaptive threshold selection by computing the optimal threshold that maximizes the between-class variance.
First start with our preparations, importing the image processing library we need:
# Import libraries
from skimage.io import imread, imshow
import matplotlib.pyplot as plt
import numpy as np
from skimage.color import rgb2gray, rgb2hsv
from skimage.filters import threshold_otsu
Load the test image sample in this section, as follows:
# Display the image that we will be playing with
original_image = imread('plants.jpg')
plt.figure(figsize=(20,20))
plt.imshow(original_image)
plt.title('Original Image', fontsize=20, weight='bold')
plt.axis('off')
plt.show()
The result is as follows:
3. Fixed threshold segmentation
When we want to binarize an image, we perform a two-step process that converts the image to grayscale, then sets any threshold we want (usually 0.50) to convert it to a binary image:
# Convert the image to grayscale
gray_image = rgb2gray(original_image)
plt.figure(figsize=(20,20))
plt.imshow(gray_image, cmap='gray')
plt.title('Grayscale Image', fontsize=20, weight='bold')
plt.axis('off')
plt.show()
The grayscale image is obtained, and the results are as follows:
Select a fixed threshold and perform binarization. The code is as follows:
# Convert the grayscale image to binary image using threshold=0.50
threshold = 0.5
binary_image = gray_image<threshold
plt.figure(figsize=(20,20))
plt.imshow(binary_image, cmap='gray')
plt.title('Binary Image', fontsize=20, weight='bold')
plt.axis('off')
plt.show()
The effect of obtaining the binary image is as follows:
Observing the above picture, we can find that for the image with obvious foreground and background discrimination, we can easily get the foreground object we need by setting a fixed threshold.
4. Adaptive Threshold Segmentation
Note how the brighter parts of the image above are darkened because we used a threshold of 0.50. But in Otsu's approach, we don't have to manually set the threshold, we can threshold_otsulet Otsu handle this work for us by calling a function, like so:
# Use threshold_otsu to automatically calculate the optimal threshold
threshold = threshold_otsu(gray_image)
print(f"Otsu's Threshold: {threshold:.2f}")
binary_image_otsu = gray_image < threshold
plt.figure(figsize=(20,20))
plt.imshow(binary_image_otsu, cmap='gray')
plt.title("Binary Image using Otsu's Method", fontsize=20, weight='bold')
plt.axis('off')
plt.show()
The result is as follows:
Output:
Otsu's Threshold: 0.67
The effect of getting a binary image is as follows:
Do you see the difference? We don't have to manually do trial and error with the threshold in order to get a better binarized image. Thanks to otsu's method, it saved us a lot of trial and error time!
5. Color Image Segmentation
Assuming we only want to isolate cyan plants in an image, it's time for another exciting technique - color image segmentation! This approach is an extension of binarization, but there is a caveat: we use color information to separate objects, not pixel value intensities. This method is especially useful when the objects of interest in the image have distinct and distinct colors.
By convention, we still import our sample image first, the code is as follows:
# Display original image
original_image = imread('plants.jpg')
plt.figure(figsize=(20,20))
imshow(original_image)
plt.title('Original Image', fontsize=20, weight='bold')
plt.axis('off')
plt.show()
The result is as follows:
6. Image segmentation based on RGB color space
Looking at the image above, in order to isolate only the lush greenery, we can perform a simple segmentation operation. If the green channel value is greater than the red and blue channel values, we can use the following comparison operator to indicate that the pixel is green, the code is as follows:
# Read image using skimage
original_image = imread('plants.jpg')
# Create subplot of 1x2
fig, ax = plt.subplots(1, 2, figsize=(20, 20))
# Plot original image
ax[0].imshow(original_image)
ax[0].set_title('Original Image', fontsize=20, weight='bold')
ax[0].axis('off')
# Get red, green, and blue channels
r = original_image[:, :, 0]
g = original_image[:, :, 1]
b = original_image[:, :, 2]
# Create a mask for green color
mask = (g > r) & (g > b) # adjust these values depending on what you consider to be 'green'
# Create a new image
new_img = original_image.copy()
# Apply mask to all channels
new_img[:, :, 0] = new_img[:, :, 0] * mask
new_img[:, :, 1] = new_img[:, :, 1] * mask
new_img[:, :, 2] = new_img[:, :, 2] * mask
# Plot the green segmented image
ax[1].imshow(new_img)
ax[1].set_title('Green Segmented Image', fontsize=20, weight='bold')
ax[1].axis('off')
# Display the subplot
plt.tight_layout()
plt.show()
The result is as follows:
7. Beautification effect
Observe the above right picture, although we can roughly get the segmentation result we want, but a lot of useless white marks are left in the above picture carefully. Essentially white is the color where red, green and blue are at their peak. So, to block these unwanted factors, we can add a white mask to the code as follows:
# Create a mask for white color
white_threshold = 180 # adjust this depending on what you consider to be 'white'
white_mask = (r > white_threshold) & (g > white_threshold) & (b > white_threshold)
# Combine the green and white masks
mask = green_mask & ~white_mask # ~ is the NOT operator
# Create a new image and apply mask
new_img = original_image.copy()
# Apply mask to all channels
new_img[:,:,0] = new_img[:,:,0] * mask
new_img[:,:,1] = new_img[:,:,1] * mask
new_img[:,:,2] = new_img[:,:,2] * mask
# Plot the green segmented image
ax[1].imshow(new_img)
ax[1].set_title('Green Segmented Image', fontsize=20, weight='bold')
ax[1].axis('off');
The result is as follows:
8. Image segmentation based on HSV color space
As mentioned above, sometimes it's easier to split colors in other color spaces like HSV (Hue, Saturation, Value). In the HSV color space, different colors are arranged on a circle (hue), so it might be easier to pick a specific color.
Let's display the hue, saturation and value of the original image:
# Read image using skimage
original_image = imread('plants.jpg')
# Convert the image to HSV color space
hsv_img = rgb2hsv(original_image)
fig, ax = plt.subplots(1, 3, figsize=(20,20))
ax[0].imshow(hsv_img[:,:,0], cmap='hsv')
ax[0].set_title('Hue', fontsize=20)
ax[1].imshow(hsv_img[:,:,1], cmap='hsv')
ax[1].set_title('Saturation', fontsize=20)
ax[2].imshow(hsv_img[:,:,2], cmap='hsv')
ax[2].set_title('Value', fontsize=20);
The result is as follows:
9. Image segmentation based on H tone
Since it's hard to see the difference in intensity values using the above plot, let's use colorbara function to analyze, the code is as follows:
plt.imshow(hsv_img[:,:,0], cmap='hsv')
plt.colorbar()
Here's the result:
As you can see, orange is between 0 and 0.05, so let's use these values as thresholds:
# Read image using skimage
original_image = imread('plants.jpg')
# Convert the image to HSV color space
hsv_img = rgb2hsv(original_image)
# Create a mask for orange color
# Hue for orange is roughly in the range of 0 - 0.05
# We can play around these values to adapt to our specific color requirement
mask = (hsv_img[:,:,0] > 0) & (hsv_img[:,:,0] < 0.05)
# create a new image and apply mask
new_img = original_image.copy()
new_img[:,:,0] = new_img[:,:,0] * mask
new_img[:,:,1] = new_img[:,:,1] * mask
new_img[:,:,2] = new_img[:,:,2] * mask
# plot the original and segmented images side by side
fig, ax = plt.subplots(1, 2, figsize=(20,10))
ax[0].imshow(original_image)
ax[0].set_title('Original Image', fontsize=20, weight='bold')
ax[0].axis('off')
ax[1].imshow(new_img)
ax[1].set_title('Orange Segmented Image', fontsize=20, weight='bold')
ax[1].axis('off')
plt.show()
The results are as follows:
Looking at the image above, we have successfully segmented orange plants with bright hues in the plant image.
10. Summary
Wow, what a ride we had today! Our in-depth study on image segmentation undoubtedly provides us with effective methods on how to extract meaningful information from images. I hope that you can enter the mysterious field of image processing through the above cases, and lament the charm of related algorithms.