Article directory
1: Regional division
Area segmentation : It is generally believed that pixels in the same area have certain similarities , such as grayscale, color, texture, etc. Area segmentation is a technology for image segmentation based on the differences in characteristics between a specific area and other background areas.
- regional growth
- Region merger
- regional fragmentation
- Region merge and split
(1) Regional growth
A: Principle
Region growing : It is an image segmentation method based on pixel similarity , aiming to gradually merge pixels with similar characteristics into larger regions . This method starts from one or more seed pixels and eventually forms a series of connected regions by gradually adding adjacent pixels that are similar to the current region. The region growing method is suitable for segmenting uniform and continuous regions, but may not be effective when faced with weak textures, noise, or blurred boundaries. The algorithm steps are as follows
- Select seed points : First select one or more seed pixels from the image as the starting point. Usually, the seed point can be manually selected or automatically determined according to some specific rules (it can be a single pixel or a sub-region including several pixels, selected based on the specific problem and using prior knowledge)
- Define similarity criteria : Determine similarity measures between pixels, such as gray value, color, texture, etc. Usually, adjacent pixels whose similarity to the seed pixel is higher than a certain threshold are selected.
- Seed expansion : Starting from the seed point, adjacent pixels are gradually added to the current area according to the similarity criterion. You can use recursion, queues, etc. to expand the seed area
- Determine the stopping conditions : In the process of region growing, it is necessary to define the stopping conditions, that is, under what circumstances the merging will stop. Common stopping conditions include reaching a certain region size, similarity measure falling below a threshold, etc.
- Iteration : Repeat the process of seed expansion and stopping condition determination until all regions are completely segmented.
The advantage of the region growing method is that it is simple and easy to use and can be applied to many image segmentation problems without the need for pre-training. However, it is sensitive to noise, weak textures, and unclear boundaries, which may lead to incorrect merging or segmentation. In order to overcome these problems, you can consider introducing more complex region segmentation algorithms, such as graph cut-based methods, deep learning methods, etc., to obtain more accurate and stable segmentation results.
B: Example
As in the following example
- Seed points : (2,2)
- Similarity criterion : The gray value difference is less than 2
- Neighborhood selection : 4 areas
C: program
As follows: perform region growing on the image. Interactively select seeds. The growth criterion adopts "the average grayscale difference between the pixel to be measured and the area is less than 40". The growth is within an 8-neighborhood range. The stop growth condition is area saturation.
matlab implementation :
clear,clc,close all;
Image=im2double(imread('lotus1.jpg'));
[height,width,channel]=size(Image);
if channel==3
Image=rgb2gray(Image);
end
figure,imshow(Image);
% Image=[1 0 4 6 5 1;1 0 4 6 6 2;0 1 5 5 5 1;0 0 5 6 5 0;0 0 1 6 0 1;1 0 1 2 1 1];
% [height,width,channel]=size(Image);
% figure,imshow(Image);
[seedx,seedy,button] = ginput(1);
seedx=round(seedx);
seedy=round(seedy);
region=zeros(height,width);
region(seedy,seedx)=1;
region_mean=Image(seedy,seedx);
region_num=1;
flag=zeros(height,width);
flag(seedy,seedx)=1;
neighbor=[-1 -1;-1 0;-1 1;0 -1;0 1;1 -1;1 0;1 1];
for k=1:8
y=seedy+neighbor(k,1);
x=seedx+neighbor(k,2);
waiting(k,:)=[y,x];
flag(y,x)=2;
end
pos=1;
len=length(waiting);
while pos<len
len=length(waiting);
current=waiting(pos,:);
pos=pos+1;
pixel=Image(current(1),current(2));
pdist=abs(pixel-region_mean);
if pdist<40/255
region(current(1),current(2))=1;
region_mean=region_mean*region_num+pixel;
region_num=region_num+1;
region_mean=region_mean/region_num;
for k=1:8
newpoint=current+neighbor(k,:);
if newpoint(1)>0 && newpoint(1)<=height && newpoint(2)>0 && newpoint(2)<width && flag(newpoint(1),newpoint(2))==0
waiting(end+1,:)=newpoint;
flag(newpoint(1),newpoint(2))=2;
end
end
end
end
figure,imshow(region),title(' ');
imwrite(region,'regiongrow.jpg');
python implementation :
import cv2
import numpy as np
import matplotlib.pyplot as plt
# 读取图像
image = cv2.imread('lotus1.jpg')
height, width, channel = image.shape
if channel == 3:
image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
plt.imshow(image, cmap='gray')
plt.show()
# [seedx, seedy, button] = ginput(1) 在这里用鼠标点击代替，获取种子点的位置
seedx, seedy = np.round(plt.ginput(1)[0])
seedx, seedy = int(seedx), int(seedy)
region = np.zeros((height, width))
region[seedy, seedx] = 1
region_mean = image[seedy, seedx]
region_num = 1
flag = np.zeros((height, width))
flag[seedy, seedx] = 1
neighbor = np.array([[-1, -1], [-1, 0], [-1, 1], [0, -1], [0, 1], [1, -1], [1, 0], [1, 1]])
waiting = []
for k in range(8):
y = seedy + neighbor[k, 0]
x = seedx + neighbor[k, 1]
waiting.append([y, x])
flag[y, x] = 2
pos = 0
while pos < len(waiting):
current = waiting[pos]
pos += 1
pixel = image[current[0], current[1]]
pdist = abs(pixel - region_mean)
if pdist < 40 / 255:
region[current[0], current[1]] = 1
region_mean = region_mean * region_num + pixel
region_num += 1
region_mean /= region_num
for k in range(8):
newpoint = current + neighbor[k]
if 0 < newpoint[0] <= height and 0 < newpoint[1] < width and flag[newpoint[0], newpoint[1]] == 0:
waiting.append(newpoint)
flag[newpoint[0], newpoint[1]] = 2
plt.imshow(region, cmap='gray')
plt.title('')
plt.show()
cv2.imwrite('regiongrow.jpg', region * 255)
(2) Regional merger
A: Principle
Region merging : It is an image segmentation technique whose main goal is to generate larger image regions with similar characteristics by merging adjacent pixels or regions . Unlike region growing, region merging is a bottom-up method that starts from a single pixel or small region and gradually merges adjacent regions until certain predefined merging conditions are met. The algorithm steps are as follows
- Initialization : Treat each pixel or small area as an independent area.
- Similarity measure : Define a metric to measure the similarity of two regions, which can include features such as pixel value, color, texture, shape, etc. A lower similarity value means the regions are more similar.
- Merge condition : Set a merge condition to determine whether two adjacent regions should be merged based on the similarity measure. This can be a threshold, meaning that only regions with similarities above the threshold can be merged.
- Region merging : Starting from a small region, iterate through each pixel or region of the image, checking for similarity with neighboring regions. If the similarity of two adjacent regions is higher than the merge condition, they are merged into a larger region.
- Iteration : Multiple iterations may be required to reach the final merged state because merging one region may affect the merging conditions of other regions
The advantage of region merging is that it can handle noise and boundary blur while producing smoother segmentation results. However, the choice of merging conditions and the implementation of the algorithm will affect the final segmentation effect. In addition, the region merging method may ignore some details and texture information, so fine segmentation results may not be obtained in some cases. Region merging is often used in medical image analysis, natural scene analysis, remote sensing image interpretation and other fields in practical applications. It can be a useful tool especially when larger region segmentations need to be generated.
B: Example
C: program
Split the small image in the example into two regions by region merging
- Initialization : Each pixel is a small area; the similarity criterion adopts: the mean gray level difference of adjacent areas ≤ 2; the first point in the upper left corner is set to area 1, and the rest are 0, indicating that it is not marked.
- Scan the image for the first time : From left to right, from top to bottom, determine the grayscale distance of each point to its upper left, upper, and left neighboring points. If the smallest of the three distances meets the merging rules, the corresponding neighbor point will be assigned a mark. The current point; if there is no similar point, give the current point a new mark
- Scan the image again : If the marks on a certain pixel point and the left neighbor point are inconsistent, but the mark of the current point is consistent with one of the neighbor points, then determine whether the two areas are the same, and if so, modify the marks of the two areas to be smaller one, that is, regional merging
matlab implementation :
clear,clc,close all;
Image=[1 0 4 6 5 1;1 0 4 6 6 2;0 1 5 5 5 1;0 0 5 6 5 0;0 0 1 6 0 1;1 0 1 2 1 1];
[height,width,channel]=size(Image);
flag=zeros(height,width);
thresh=2;
neighbor=[-1 -1;-1 0;0 -1];
flag(1,1)=1;
number=1;
for j=1:height
for i=1:width
pdist=[300 300 300];
for k=1:3
y=j+neighbor(k,1);
x=i+neighbor(k,2);
if x>=1 && y>=1
pdist(k)=abs(Image(j,i)-Image(y,x));
end
end
[mindist,pos]=min(pdist(:));
if mindist<=thresh
y=j+neighbor(pos,1);
x=i+neighbor(pos,2);
if flag(y,x)
flag(j,i)=flag(y,x);
end
elseif mindist~=300
number=number+1;
flag(j,i)=number;
end
end
end
for j=2:height
for i=2:width
if flag(j-1,i)~=flag(j,i-1) && (flag(j,i)==flag(j-1,i) || flag(j,i)==flag(j,i-1))
pdist=abs(Image(j-1,i)-Image(j,i-1));
if pdist<=thresh
minv=min(flag(j-1,i),flag(j,i-1));
maxv=max(flag(j-1,i),flag(j,i-1));
flag(flag==maxv)=minv;
end
end
end
end
python implementation :
import numpy as np
import matplotlib.pyplot as plt
Image = np.array([[1, 0, 4, 6, 5, 1],
[1, 0, 4, 6, 6, 2],
[0, 1, 5, 5, 5, 1],
[0, 0, 5, 6, 5, 0],
[0, 0, 1, 6, 0, 1],
[1, 0, 1, 2, 1, 1]])
height, width = Image.shape
flag = np.zeros((height, width))
thresh = 2
neighbor = np.array([[-1, -1], [-1, 0], [0, -1]])
flag[0, 0] = 1
number = 1
for j in range(height):
for i in range(width):
pdist = [300, 300, 300]
for k in range(3):
y = j + neighbor[k, 0]
x = i + neighbor[k, 1]
if x >= 0 and y >= 0:
pdist[k] = abs(Image[j, i] - Image[y, x])
mindist, pos = min((val, idx) for (idx, val) in enumerate(pdist))
if mindist <= thresh:
y = j + neighbor[pos, 0]
x = i + neighbor[pos, 1]
if flag[y, x]:
flag[j, i] = flag[y, x]
elif mindist != 300:
number += 1
flag[j, i] = number
for j in range(1, height):
for i in range(1, width):
if flag[j - 1, i] != flag[j, i - 1] and (flag[j, i] == flag[j - 1, i] or flag[j, i] == flag[j, i - 1]):
pdist = abs(Image[j - 1, i] - Image[j, i - 1])
if pdist <= thresh:
minv = min(flag[j - 1, i], flag[j, i - 1])
maxv = max(flag[j - 1, i], flag[j, i - 1])
flag[flag == maxv] = minv
plt.imshow(flag, cmap='tab20')
plt.title('')
plt.show()
(3) Regional split
A: Principle
Region splitting : It is an image segmentation technology. Contrary to region merging, it starts from an overall region and gradually divides the region into smaller sub-regions until certain predetermined segmentation conditions are met . Region splitting technology usually adopts a top-down approach, starting from a whole and gradually decomposing it into sub-regions until each sub-region meets specific conditions. The algorithm steps are as follows
- Initialization : Treat the entire image or the entire area as an initial area.
- Division condition : Set a division condition to determine whether an area should be divided into sub-areas based on certain specific criteria. This may involve features such as pixel values, color, texture, shape, etc.
- Region splitting : Starting from the initial region, check whether the region meets the division conditions. If the conditions are met, the area is divided into smaller sub-areas.
- Recursive : For each sub-region, if the conditions for continued division are met, the sub-region will continue to be recursively divided until the division conditions are no longer met.
- Repeat : Repeat the region splitting and recursive steps until all regions that meet the conditions are divided
The advantage of region splitting is that it can start from the whole and gradually decompose it into sub-regions with similar characteristics, thereby obtaining more accurate segmentation results. However, the choice of dividing conditions and the implementation of the algorithm will affect the final segmentation effect. In addition, the region splitting algorithm may produce too many small regions in some cases, leading to over-segmentation problems. Region splitting is usually used in medical image analysis, object detection, image segmentation and other fields in practical applications. Region splitting can be a useful tool when you need to segment an image into multiple regions with similar characteristics
B: Example
1: Initialization and determination of criteria and methods
- The difference between the maximum gray value and the minimum gray value in the area ≤ \leq≤ 2
- Use the split method of dividing into four
2: Use the split method of dividing it into four
Calculate the maximum and minimum grayscale differences for four small areas respectively, and compare them with threshold 2. Each area needs to be split.
And so on, until all areas can no longer be divided
C: program
as follows
matlab implementation :
Image=imread('cameraman.jpg');
S=qtdecomp(Image,0.27);
blocks=repmat(uint8(0),size(S));
for dim=[256 128 64 32 16 8 4 2 1]
numblocks=length(find(S==dim));
if(numblocks>0)
values=repmat(uint8(1),[dim dim numblocks]);
values(2:dim,2:dim,:)=0;
blocks=qtsetblk(blocks,S,dim,values);
end
end
python implementation :
import cv2
import numpy as np
import matplotlib.pyplot as plt
# 读取图像
Image = cv2.imread('cameraman.jpg', cv2.IMREAD_GRAYSCALE)
# 进行 Quadtree 分解
S, _ = cv2.qr(Image, 0.27)
blocks = np.zeros_like(S, dtype=np.uint8)
for dim in [256, 128, 64, 32, 16, 8, 4, 2, 1]:
numblocks = np.sum(S == dim)
if numblocks > 0:
values = np.ones((dim, dim, numblocks), dtype=np.uint8)
values[1:dim, 1:dim, :] = 0
blocks = cv2.qt_set_block(blocks, S, dim, values)
plt.imshow(blocks, cmap='gray')
plt.title('')
plt.show()
(4) Regional division and merger
A: Principle
Region splitting and merging : It is an image segmentation method that combines the ideas of region segmentation and region merging, aiming to segment the image into regions with similar characteristics through a bottom-up method. This method first divides the image into initial small areas, and then gradually merges adjacent areas according to certain merging criteria to obtain larger and more consistent areas. The goal of the region splitting and merging method is to generate accurate segmentation results based on making full use of local consistency. The algorithm steps are as follows
- Initial segmentation : Divide the entire image into initial small areas. Each area can be a single pixel or a larger local area.
- Splitting process : First, split judgment is performed on each area. The judgment conditions can be similarity between pixels, color consistency, etc. If a region does not meet the splitting conditions, then the region will no longer be split, otherwise the region will continue to be split into smaller sub-regions.
- Merging process : In the small area obtained by splitting, determine whether the adjacent areas meet the merging conditions. The merging conditions can be that the similarity is higher than the threshold, etc. If adjacent regions meet the merging conditions, merge them into a larger region
- Iteration : The splitting and merging process is performed repeatedly until no further splitting and merging can be performed, generating the final segmentation result.
The advantage of region splitting and merging is that it can process regions of different sizes in a bottom-up process, thereby obtaining a larger range of segmentation results while retaining local consistency. However, the efficiency of this method may be affected by the choice of region segmentation and merging conditions and the implementation of the algorithm. Region splitting and merging are widely used in practical applications in medical image analysis, remote sensing image interpretation, natural scene analysis and other fields, and are especially suitable for image segmentation tasks in complex scenes.
B: Example
2: Image segmentation based on clustering
(1) Principle
Image segmentation based on clustering : Think of image segmentation as a problem of classifying pixels . The pixels are represented as points in the feature space. The clustering algorithm is used to divide these points into different categories. The corresponding original image is to group the pixels. , use "connected component markers" to find connected regions after grouping. How to represent pixels as points in feature space
: use vectors to represent pixels or the neighborhood around pixels. The elements of the vector are features related to the pixels. Based on the specific conditions of the image, the design is based on judging the commonality of the areas to be segmented.
(2) K-means clustering
- Omitted (please check the clustering column)
(3) Procedure
as follows:
matlab implementation :
clear,clc,close all;
Image=imread('fruit.jpg');
imshow(Image);
hsv=rgb2hsv(Image);
h=hsv(:,:,1);
h(h>330/360)=0;
[N,M]=size(h);
training=h(:);
startdata = [0;60/360;120/360;180/360;240/360;300/360];
[IDX,C]= kmeans(training,6,'Start',startdata);
idbw = (IDX == 1);
template = reshape(idbw, size(h));
figure,imshow(template),title('分割后的图像');
imwrite(template,'clusterseg.jpg');
python implementation :
import cv2
import numpy as np
from sklearn.cluster import KMeans
import matplotlib.pyplot as plt
# 读取图像
Image = cv2.imread('fruit.jpg')
Image = cv2.cvtColor(Image, cv2.COLOR_BGR2RGB)
# 显示原始图像
plt.imshow(Image)
plt.title('')
plt.show()
# 转换为HSV颜色空间并提取H通道
hsv = cv2.cvtColor(Image, cv2.COLOR_RGB2HSV)
h = hsv[:, :, 0]
h[h > 330] = 0 # 将大于330的值设为0
N, M = h.shape
training = h.reshape(-1, 1)
startdata = np.array([0, 60, 120, 180, 240, 300]) / 360
# 使用K均值聚类进行分割
kmeans = KMeans(n_clusters=6, init=startdata.reshape(-1, 1), n_init=1)
kmeans.fit(training)
labels = kmeans.labels_
centers = kmeans.cluster_centers_
idbw = (labels == 0)
template = idbw.reshape((N, M))
# 显示分割后的图像
plt.imshow(template, cmap='gray')
plt.title('')
plt.show()
cv2.imwrite('clusterseg.jpg', template * 255)