The method of obtaining the data code of this article, public account: Xiao Zhang Python, backstage reply keyword: Christmas tree
Hello everyone, I am zeroing. Yesterday was Christmas. I don’t know if you guys are having a good time. Anyway, I am very happy. In the evening, my roommates all went out alone in the dormitory, doing whatever they want~ It’s really cool~
This article will use Python to realize the recognition and marking of the Christmas tree in the picture, which can be understood as object detection in computer vision. First, let me declare that there is no neural network used here. They are all traditional methods.
First look at the effect, the following is the original picture
The following is the final result of the detection:
The outline of the Christmas tree in the picture is marked with a red line, the effect looks good~, the following is the overall idea of the algorithm implementation, which is divided into three parts
1. Extract image feature points (according to image brightness, hue, saturation)
In the 6 images shown above, due to the colorful lights, the Christmas tree appears brighter and warmer in the entire picture, in contrast to the colder and blueish background;
According to the above-mentioned ideas, first extract the feature points on the Christmas tree. Here, the image is conditionally filtered from the three angles of brightness, hue, and saturation to filter out the target feature point set in the image. The selection criteria are as follows
- 1. When doing brightness screening, first convert RGB to grayscale image, and extract the area with grayscale value greater than 220 (original image standard 0-255)
- 2. Convert the image from RGB(0-255) to HSV(0-1) color space, and extract the area in HSV whose hue (hue channel) value is less than 0.2 or greater than 0.95. Less than 0.2 is to extract yellowish and reddish images in the image Feature points, greater than 0.95 corresponds to the purple-red area on the edge of the Christmas tree
- 3. In the image HSV color space, extract the parts whose saturation and value are greater than 0.7 ;
Here is a brief introduction to HSV. HSV is a color space of the picture, similar to the three channels of RGB. RGB represents the three channels of red, green and blue respectively; and HSV represents hue (hue), saturation (saturation), value ( brightness);
- Hue H: Measured by angle, the value range is 0°～360°, starting from red and counting in a counterclockwise direction, red is 0°, green is 120°, and blue is 240°. Their complementary colors are: yellow is 60°, cyan is 180°, and magenta is 300°; (this article converts 0-300 degrees to 0-1.0 range values)
- Saturation S: The value range is 0.0～1.0;
- Brightness V: The value range is 0.0 (black) to 1.0 (white).
According to the above three filters, the image processing to finally obtain a binary black and white image, here in numpy logical_andand logical_ormethods for the polymerization of the above three conditions;
As you can see from the above picture, the black points in the picture are the extracted feature points (Christmas tree). The basic outline has been out, but there will be a little noise. See Figures 2 and 4. The lighting and horizon features in the building are also Has been extracted, but these are not what we need, so the following step is needed: clustering , to remove these noise points
2. Use DBSCAN algorithm to cluster feature points
After obtaining the feature points in the previous step, the feature point set will be clustered below. Regarding the point set clustering, the DNSCAN algorithm based on spatial density is used here. This algorithm has been encapsulated in the scikit-learn package and can be called directly when used. , But because there are some parameter settings involved, two parameters need to be paid attention to when using:
eps, A parameter in the algorithm represents the maximum distance between the class and the class sample. For different data sets and distance functions, this parameter needs to be set to a different value; here is set to 0.04 times the diagonal length of the image, so that it can be adapted Large resolution pictures can also be applied to small resolution picturesmin_samples, Assuming a certain point as the center, the number of surrounding samples (including the sample itself); if the value is too small, the final category will be too much, and if the value is too large, the final category will be too few; this article is set to 10;
After the feature points are classified, the feature points of the Christmas tree are finally marked in red, and the effect is as follows:
The effect after the stroke is expanded:
It can be seen that the feature points in Figures 2, 3, and 4 are divided into two categories, marked with different colors; a conditional filter will be done later: only the category with the largest number of feature points in the picture (Christmas tree) can be selected. Noise removal in the image
3. Calculate the convex hull of the target feature point set and draw it on the original image
This last step is much simpler, with the feature point set, using the scipycalculation method of the convex hull ConvexHull package, then re-use matplotlibthe convex hull are drawn on the original
summary
Some technical points in the article are worth learning, such as the aforementioned use of hue and saturation as threshold conditions to filter feature points, and the use of DBSCAN clustering algorithm later; these Ideas are not only limited to Christmas trees, but can also be used Detect some other objects from above, but you need to think more and practice more
Finally, here is why the clustering algorithm uses DBSCAN instead of the classic KMeans; because KMeans needs to set the number of categories when classifying (the number of categories is not determined in advance), and only the Euclidean distance is used as a reference when classifying. The final classification result is not ideal, refer to the figure below
KMeans algorithm
DBSCAN algorithm
Core code used in the article
from PIL import Image
import numpy as np
import scipy
import matplotlib.colors as colors
from sklearn.cluster import DBSCAN
from math import ceil,sqrt
'''
Inputs:
rgbimg: M,N,3 numpy 包含 uint(0-255) color image
hueleftthr: Scalar constant to maximum hue in yellow-green region
huerightthr: Scalar constant to maximum allowed hue in blue-purple region
satthr: Scalar constant to select minimum allow saturation
valthre: Scalar constant to select minimum allow value
monothr: Scalar constant to select minimum allow monochrome
maxpoints: Scalar constant maximum number of pixels to forward to the DBSCAN clustering algoritm
proxthresh: Proximity threshold to use for DBSCAN, as da fraction of the diagonal size of thre image
接近阈值占图像对角线尺寸
Outputs:
borderseg: [K,2,2] Nested list containing K pairs of x- and y- pixel values for drawimg the tree border
X: [P,2] List of pixels that passed the threshold step
labels: [Q,2] List of cluster labels for points in Xslice(see below)
Xslice: [Q,2] Reduced list of pixels to be passed to DBSCAN
'''
'''实现脚本'''
def findtree(rgbimg,
hueleftthr = 0.2,
huerightthr = 0.95,
satthr =0.7,
valthr = 0.7,
monothr = 220,
maxpoints = 5000,
proxthresh = 0.04):
# 将 RGB 图像转化为 灰度图
grayimg = np.asarray(Image.fromarray(rgbimg).convert('L'))
# 将 rbg => hsv(float [0,1.0])
hsvimg = colors.rgb_to_hsv(rgbimg.astype(float)/255)
# 二值化阈值图像初始化
binimg = np.zeros((rgbimg.shape[0],rgbimg.shape[1]))
#1， heu < 0.2 or hue > 0.95(red or yellow)
#2， saturated and bright both greater than 0.7
# 满足以上条件被认为是圣诞树上的灯
boolidx = np.logical_and(
np.logical_and(
np.logical_or((hsvimg[:,:,0]<hueleftthr),
(hsvimg[:,:,0]>huerightthr)),
(hsvimg[:,:,1]>satthr)),
(hsvimg[:,:,2]>valthr))
# 找到满足 hsv 标准的像素，赋值为255
binimg[np.where(boolidx)] = 255
# 添加像素来满足garay brightness 条件
binimg[np.where(grayimg>monothr)] = 255
# 用 DBSCAN 聚类算法分割这些点
X = np.transpose(np.where(binimg==255))
Xslice = X
nsample = len(Xslice)
if nsample > maxpoints:
# 确保样本数不超过 DNSCAN 算法最大限度
Xslice = X[range(0,nsample,int(ceil(float(nsample/maxpoints))))] # 将样本每隔几个采样一次
# 将 DNSCAN 阈值接近像素单位，并运行 DBSCAN
pixproxthr = proxthresh * sqrt(binimg.shape[0]**2 + binimg.shape[1]**2) # 对角巷长*proxthresh
db = DBSCAN(eps = pixproxthr,min_samples=10).fit(Xslice) # 拟合样本
labels = db.labels_.astype(int)
# 寻找最大聚类
unique_labels = set(labels)
maxclustpt = 0
for k in unique_labels:
class_numbers = [index[0] for index in np.argwhere(labels==k)]
if(len(class_numbers) > maxclustpt):
points = Xslice[class_numbers]
hull = scipy.spatial.ConvexHull(points) # 建立凸包
maxclustpt = len(class_numbers)
borderseg = [[points[simplex,0], points[simplex,1]] for simplex in hull.simplices]
return borderseg,X,labels,Xslice
Startup script
'''
@author:zeroing
@wx公众号：小张Python
'''
from PIL import Image
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.cm as cm
from findtree import findtree
import os
path_dir = 'D:/ceshi_11/findtree'
path_list = [os.path.join(path_dir,str(i)) for i in os.listdir(path_dir)]
# 初始化figure size
fgsz = (16,8)
figthresh = plt.figure(figsize = fgsz,facecolor ='w')
figclust = plt.figure(figsize = fgsz,facecolor ='w')
figcltwo = plt.figure(figsize = fgsz,facecolor = 'w')
figborder = plt.figure(figsize = fgsz,facecolor = 'w')
figorigin = plt.figure(figsize = fgsz,facecolor = 'w')
# 每张图设置一个 窗口名
figthresh.canvas.set_window_title('Thresholded HSV and Monochrome Brightness')
figclust.canvas.set_window_title('DBSCAN Clusters (Raw Pixel Output)')
figcltwo.canvas.set_window_title('DBSCAN Clusters (Slightly Dilated for Display)')
figborder.canvas.set_window_title('Trees with Borders')
figorigin.canvas.set_window_title("Original Image")
for ii,name in enumerate(path_list):
# 打开图片
rgbimg = np.asarray(Image.open(str(name)))
# 运行脚本找到 bordeseg,X,Labels,Xslce
borderseg,X,labels,Xslice = findtree(rgbimg)
# 展示阈值分割后的图像
axthresh = figthresh.add_subplot(2,3,ii+1)
axthresh.set_xticks([])
axthresh.set_yticks([])
binimg = np.zeros((rgbimg.shape[0],rgbimg.shape[1]))
for v,h in X:
binimg[v,h] = 255 # 初步筛选之后坐标点
axthresh.imshow(binimg,interpolation = 'nearest',cmap = 'Greys')
# Display color-coded clusters
axclust = figclust.add_subplot(2,3,ii+1)
axclust.set_xticks([])
axclust.set_yticks([])
axcltwo = figcltwo.add_subplot(2,3,ii+1)
axcltwo.set_xticks([])
axcltwo.set_yticks([])
axcltwo.imshow(binimg,interpolation = 'nearest',cmap = 'Greys')
clustimg = np.ones(rgbimg.shape)
unique_labels = set(labels)
# 为每个聚类生成单个颜色
plcol = cm.rainbow_r(np.linspace(0,1,len(unique_labels)))
print('plcol',plcol)
for lbl,pix in zip(labels,Xslice):
for col,unqlbl in zip(plcol,unique_labels):
if lbl == unqlbl:
# -1 表示无聚类成员
if lbl == -1:
col = [0.0,0.0,0.0,1.0]
for ij in range(3):
clustimg[pix[0],pix[1],ij] = col[ij]
# 扩张 图像，用于更好展示
axcltwo.plot(pix[1],pix[0],'o',markerfacecolor= col,markersize = 1,markeredgecolor = col)
axclust.imshow(clustimg)
axcltwo.set_xlim(0,binimg.shape[1]-1)
axcltwo.set_ylim(binimg.shape[0],-1)
# 在原图树边缘进行绘制
axborder = figborder.add_subplot(2,3,ii+1)
axborder.set_axis_off()
axborder.imshow(rgbimg,interpolation ='nearest')
for vseg,hseg in borderseg:
axborder.plot(hseg,vseg,'g-',lw =3)
axborder.set_xlim(0,binimg.shape[1]-1)
axborder.set_ylim(binimg.shape[0],-1)
# 保存原图
origin_fig1 = figorigin.add_subplot(2, 3, ii + 1)
origin_fig1.set_axis_off()
origin_fig1.imshow(rgbimg, interpolation='nearest')
axborder.set_xlim(0, binimg.shape[1] - 1)
axborder.set_ylim(binimg.shape[0], -1)
# axborder.savefig("D:/ceshi_11/findtree/final_")
print(name,'Sucessfully find it !!!!!!!!')
plt.show()
Well, the above is the entire content of this article. If you think the content is good, please like, share, and leave a message. Finally, thank you for reading!
Reference link: https://stackoverflow.com/questions/20772893/how-to-detect-a-christmas-tree