Local image descriptor------match geotagged images

3. Match geotagged images

Geotagged images can be obtained from Panoramio, a photo-sharing service provided by Google. Like many network resources, Panoramio provides an API interface to facilitate users to access these contents using programs.

1. Experimental principle

1.1 Use local descriptor matching

Extract local descriptors from downloaded images. In this case, we will use SIFT feature descriptors. We assume that these images have been processed using SIFT feature extraction code and the features are saved in a file with the same name as the image (but with a .sift suffix instead of .jpg).

1.2 Visualizing connected images

We first define the connections between images by whether they have matching local descriptors, and then visualize these connections. To complete the visualization, we can display these images in a graph, where the edges represent connections. We will use the pydot toolkit, which is a Python interface to the powerful GraphViz graphics library. Pydot uses Pyparsing and GraphViz. To create a graph showing possible image groups, we use edges to connect corresponding image nodes if the number of matches is above a threshold. In order to get the image in the picture, you need to use the full path of the image. To make the images look prettier, we need to scale each image into thumbnail form.

2. Code implementation

import cv2 as cv
from pylab import *
import os
import pydotplus as pydot

maxsize = (100, 100)  # 定义缩略图的大小
path = r'D:\software\pycharm\PycharmProjects\computer-version\two\images'

# 读取整个文件夹的图片
def read_path(pathname):
    imgname_list = os.listdir(pathname)
    img_list = []
    i = 0
    # 图片列表
    for imgname in imgname_list:
        if imgname.endswith('.jpg'):
            img = cv.imread(pathname + '/' + imgname)
            img_n = cv.resize(img, maxsize, cv.INTER_AREA)
            filename = path + str(i) + '.png'
            cv.imwrite(filename, img_n)  # need temporary files of the right size
            i = i + 1
            print(i)
    return img_list

list = read_path(r'course/computer_vision/data/all_data')
print(list)

# 读取整个文件夹的图片
def read_path(pathname):
    imgname_list = os.listdir(pathname)
    img_list = []
    # 图片列表
    for imgname in imgname_list:
        if imgname.endswith('.jpg'):
            img = cv.imread(pathname + '/' + imgname)
            img_list.append(img)
    return img_list

img_list = read_path(r'D:\software\pycharm\PycharmProjects\computer-version\two\images')
nbr_images = len(img_list)
match_scores = zeros((nbr_images, nbr_images))

for i in range(nbr_images):
    for j in range(i, nbr_images):  # only compute upper triangle
        print('comparing ', i, j)
        sift = cv.xfeatures2d.SIFT_create()
        kp1, des1 = sift.detectAndCompute(img_list[i], None)
        kp2, des2 = sift.detectAndCompute(img_list[j], None)
        # BFMatch匹配
        bf = cv.BFMatcher(cv.NORM_L2)
        matches = bf.knnMatch(des1, des2, k=2)
        # 储存差距小的优秀匹配点
        goodMatches = []
        for m, n in matches:
            if m.distance < 0.5 * n.distance:
                goodMatches.append(m)
        # 计算优秀匹配点的和
        nbr_matches = len(goodMatches)
        # 向match_scores赋值
        print('number of matches = ', nbr_matches)
        match_scores[i, j] = nbr_matches

# 复制
for i in range(nbr_images):
    for j in range(i + 1, nbr_images):  # no need to copy diagonal 不用复制自我匹配的对角线
        match_scores[j, i] = match_scores[i, j]
# 可视化
threshold = 2  #  至少2个以上匹配点就可以算是联系
g = pydot.Dot(graph_type='graph')  # 不需要有向图
maxsize = (100, 100)  # 定义缩略图的大小
path = r'D:\software\pycharm\PycharmProjects\computer-version\two\images'
#两两配对
for i in range(nbr_images):
    for j in range(i + 1, nbr_images):
        if match_scores[i, j] > threshold:
            filename = path + str(i) + '.png'
            g.add_node(pydot.Node(str(i), fontcolor='transparent', shape='rectangle', image=filename))
            filename = path + str(j) + '.png'
            g.add_node(pydot.Node(str(j), fontcolor='transparent', shape='rectangle', image=filename))
            g.add_edge(pydot.Edge(str(i), str(j)))
#绘制S地理标记SIFT匹配图
g.write_jpg('sift.jpg')

3.Result display

Analysis: This data set selected three different scenes for scale and angle transformation. From the experimental results, it can be seen that SIFT has successfully combined images of the same scene based on local descriptors, but there are also cases of incorrect matching. This experiment verifies that SIFT features are invariant to scale, rotation, and brightness. However, it can be seen from the experimental process and experimental results that SIFT has shortcomings: the larger the picture or the more complex the picture, the longer the running time of the program will be. Too time consuming.