python Computer Vision - local image descriptors

1. Harris corner detection

1.1 Harris corner detection principle

Harris corner detection algorithm is the easiest one corner detection method. The basic idea is to use a sliding window fixed in an arbitrary direction on the image, comparison with the former two cases slide, the degree of gradation of the pixels in the window after the change slide, the slide in any direction if present, have a greater gray change, then we can consider the presence of corner points of the window.
Mathematically to describe the corner features:

When the window occurs [u, v] is moved, then the front slide and the rear slide gradation change corresponding to the pixel window is described as follows:

Equation explained:
[u, v] is the offset of the window
(x, y) is the window corresponding to the pixel coordinate position, how big window, there are many positions
w (x, y) is the window function, the simplest case is that all the pixels within the window corresponding to the weighting coefficient w are 1. But sometimes, we will w (x, y) is set to the window function of the origin of a bivariate normal distribution center. If the window is the center point of the corner point, before moving and after moving, the gray-scale variation should be most intense point, so that the weighting coefficient may be set point larger, showing the window moves, the larger the intensity change contribution point ; from the center of the window (corner) distant points, these points gradation change almost flat, the weight coefficient of these points can be set to a small point, the point to show gray scale smaller contribution, we natural to think Gaussian function represented using a binary window function, the window function generally has the following two forms:

according to the above expression, when the sliding window is in the flat area, it is conceivable that the gradation does not change, then E (u, v) = 0; if the window is on the rich textures of the regions than the slide, the gradation changes greatly. The final position calculation algorithm idea is that when large changes occur corresponding to the gradation, of course, the larger the sliding direction of the pointer is arbitrary, does not refer to a single direction.

Reference blog View Comments

1.2 Harris corner detection code examples

# -*- coding: utf-8 -*-
from pylab import *
from PIL import Image
from PCV.localdescriptors import harris

"""
Example of detecting Harris corner points (Figure 2-1 in the book).
"""

# 读入图像
im = array(Image.open('C://Users//Garfield//Desktop//01.jpg').convert('L'))

# 检测harris角点
harrisim = harris.compute_harris_response(im)

# Harris响应函数
harrisim1 = 255 - harrisim

figure()
gray()

#画出Harris响应图
subplot(141)
imshow(harrisim1)
print(harrisim1.shape)
axis('off')
axis('equal')

threshold = [0.01, 0.05, 0.1]
for i, thres in enumerate(threshold):
    filtered_coords = harris.get_harris_points(harrisim, 6, thres)
    subplot(1, 4, i+2)
    imshow(im)
    print(im.shape)
    plot([p[1] for p in filtered_coords], [p[0] for p in filtered_coords], '*')
    axis('off')

#原书采用的PCV中PCV harris模块
#harris.plot_harris_points(im, filtered_coords)

# plot only 200 strongest
# harris.plot_harris_points(im, filtered_coords[:200])

show()

Code run shot:

Minor changes to the code above, the effect of corner detection clearer:
Reference blog

# -*- coding: utf-8 -*-
from pylab import *
from PIL import Image
from PCV.localdescriptors import harris

# 读入图像
im = array(Image.open('C://Users//Garfield//Desktop//01.jpg').convert('L'))

# 检测harris角点
harrisim = harris.compute_harris_response(im)

# Harris响应函数
harrisim1 = 255 - harrisim

figure()
gray()

#画出Harris响应图
subplot(121)
suptitle("Harris corners")
imshow(harrisim1)
print (harrisim1.shape)
axis('off')
axis('equal')

threshold = [0.01, 0.05, 0.1]
for i, thres in enumerate(threshold):
    filtered_coords = harris.get_harris_points(harrisim, 6, thres)

subplot(1, 2, 2)
imshow(im)
print (im.shape)
plot([p[1] for p in filtered_coords], [p[0] for p in filtered_coords], '+c')
axis('off')
show()

Comparative Experiment 2. Harris corner detection based on different kinds of images made

2.1 a flat image Harris corner detection results

Code:

# -*- coding: utf-8 -*-
from pylab import *
from PIL import Image
from PCV.localdescriptors import harris

# 读入图像
im = array(Image.open('C://Users//Garfield//Desktop//p1.jpg').convert('L'))

# 检测harris角点
harrisim = harris.compute_harris_response(im)

# Harris响应函数
harrisim1 = 255 - harrisim

figure()
gray()

#画出Harris响应图
subplot(121)
suptitle("Harris corners")
imshow(harrisim1)
print (harrisim1.shape)
axis('off')
axis('equal')

threshold = [0.01, 0.05, 0.1]
for i, thres in enumerate(threshold):
    filtered_coords = harris.get_harris_points(harrisim, 6, thres)

subplot(1, 2, 2)
imshow(im)
print (im.shape)
plot([p[1] for p in filtered_coords], [p[0] for p in filtered_coords], '+c')
axis('off')
show()

1. Right side:

2. The left side:
   
3. Underneath
   
4. positive:
   
5. Dim near the top:
   

2.1.1 Analysis of experimental results

Can be seen that corner of the intermediate region are not changed obviously, will transform the transformation angle corner angle change with the outside disappeared or add new corners.

harris corner detection operator changes the brightness and contrast are not sensitive

这是因为在进行Harris角点检测时，使用了微分算子对图像进行微分运算，而微分运算对图像密度的拉升或收缩和对亮度的抬高或下降不敏感。换言之，对亮度和对比度的仿射变换并不改变Harris响应的极值点出现的位置，但是，由于阈值的选择，可能会影响角点检测的数量。

2.2 边缘丰富图像的Harris角点检测实验结果

1. 蓝天白云下：

2. 黄昏时：
   
3. 光线暗时
   
4. 换个角度
   
5. 旋转
   

2.2.1 实验结果分析

对比b2正常角度与旋转变化的结果可以证实

Harris角点检测算子具有旋转不变性

Harris角点检测算子使用的是角点附近的区域灰度二阶矩矩阵。而二阶矩矩阵可以表示成一个椭圆，椭圆的长短轴正是二阶矩矩阵特征值平方根的倒数。当特征椭圆转动时，特征值并不发生变化，所以判断角点响应值RR也不发生变化，由此说明Harris角点检测算子具有旋转不变性。

拓展:求解E(u,v)

于是对于局部微小的移动量 [u,v]，可以近似得到下面的表达：

其中M是 2*2 矩阵，可由图像的导数求得:




Harris角点检测通过对窗口内的每个像素的x方向上的梯度与y方向上的梯度进行统计分析。这里以Ix和Iy为坐标轴，因此每个像素的梯度坐标可以表示成(Ix,Iy)。针对平坦区域，边缘区域以及角点区域三种情形进行分析：

平坦区域上的每个像素点所对应的(IX,IY)坐标分布在原点附近，其实也很好理解，针对平坦区域的像素点，他们的梯度方向虽然各异，但是其幅值都不是很大，所以均聚集在原点附近；边缘区域有一坐标轴分布较散，至于是哪一个坐标上的数据分布较散不能一概而论，这要视边缘在图像上的具体位置而定，如果边缘是水平或者垂直方向，那么Iy轴方向或者Ix方向上的数据分布就比较散；角点区域的x、y方向上的梯度分布都比较散。

虽然一般利用E(u,v)来描述角点的基本思想，然而最终使用的还是矩阵M。矩阵M形式跟协方差矩阵形式很像,但不同之处在于一般协方差矩阵对应维的随机变量需要减去该维随机变量的均值，但矩阵M中并没有这样做，所以在矩阵M里，先进行各维的均值化处理，那么各维所对应的随机变量的均值为0，协方差矩阵就大大简化了，简化的最终结果就是矩阵。
如果对协方差矩阵M进行对角化，很明显，特征值就是主分量上的方差。如果存在两个主分量所对应的特征值都比较大，说明像素点的梯度分布比较散，梯度变化程度比较大，符合角点在窗口区域的特点；如果是平坦区域，那么像素点的梯度所构成的点集比较集中在原点附近，因为窗口区域内的像素点的梯度幅值非常小，此时矩阵M的对角化的两个特征值比较小；如果是边缘区域，在计算像素点的x、y方向上的梯度时，边缘上的像素点的某个方向的梯度幅值变化比较明显，另一个方向上的梯度幅值变化较弱，其余部分的点都还是集中原点附近，这样M对角化后的两个特征值理论应该是一个比较大，一个比较小，当然对于边缘这种情况，可能是呈45°的边缘，致使计算出的特征值并不是都特别的大，总之跟含有角点的窗口的分布情况还是不同的。

2.3 角点丰富图像的Harris角点检测实验结果

1. 正面暖光
   

2. 近
   

3. 侧面
   

4. 正面白光
   

5. 左侧

2.3.1 实验结果分析

可以明显看到侧面相当于正面来说角点数大幅度减少,经过实验还可得知一定程度的光照可以使得角点更易被捕捉,而太明亮或者太暗都会使实验结果不理想,甚至无法识别到角点,但角点的关键点并不会改变。
同时可以看到远处与近处图片的对比,关键点明显改变,可证实:

Harris角点检测算子不具有尺度不变性

正如下图所示，当右图被缩小时，在检测窗口尺寸不变的前提下，在窗口内所包含图像的内容是完全不同的。左侧的图像可能被检测为边缘或曲线，而右侧的图像则可能被检测为一个角点。

Reference blog https://blog.csdn.net/lwzkiller/article/details/54633670
reference blog https://blog.csdn.net/weixin_41923000/article/details/88631944
reference blog https://blog.csdn.net/ qq_41598072 / article / details / 83651629