Multiple image panorama stitching code modification

Enterprise 2023-05-18 12:10:58 views: null

foreword

Reference code: github code address

1. Create a virtual environment

conda create -n stitching python=3.7.0

insert image description here

2. Activate the virtual environment

conda activate stitching

3. Install opencv and corresponding packages

3.1 View the installable opencv version

The latest version of the relevant opencv installation can be seen opencv installation

pip install opencv-python==

insert image description here

to install

pip install opencv-python==3.4.2.16

carry out testing
insert image description here

3.2 Install the corresponding package

pip install imutils

3.3 Install opencv-contrib-python

The solution to the error
insert image description here
: add a version of the matching contrib package

pip install opencv-contrib-python==3.4.2.16

3.4 matplotlib installation

pip install matplotlib

3.5 PCV installation

You can refer to PCV installation

  1. Enter the corresponding folder clone PCV code
git clone https://github.com/Li-Shu14/PCV.git
  1. Install
    here because I installed it in the windows environment
    First change the drive letter to your corresponding drive such as D:
    Enter the folder cd xxx/PCV
    insert image description here
python setup.py install

3.5 scipy installation

Returning to the reference code, an error occurred when running the program

ModuleNotFoundError: No module named 'scipy'

Install in the corresponding environment

pip install scipy

4. Error record

4.1 print error

SyntaxError: Missing parentheses in call to 'print'. Did you mean print('warp - right')?

insert image description here
The previous PCV was applied to python2, and now there is a python3-based PCV github updated for python3

Refer to PCV installation to install based on python3

4.2 matplotlib.delaunay reports an error

ModuleNotFoundError: No module named 'matplotlib.delaunay'

For the change method, please refer to matplotlib.delaunay to report an error
and then re-install (you cannot directly enter the corresponding file here, so you need to re-install after modification. I have an error in the installation here. I directly deleted the virtual environment and recreated it, and then installed it)

4.3 Unable to generate .sift file

insert image description here

  File "D:\anaconda\envs\stitching\lib\site-packages\numpy\lib\_datasource.py", line 533, in open
    raise IOError("%s not found." % path)
OSError: ./test/1027-1.sift not found.

Reference link:
OSError: sift not found problem solving
image stitching reference link

  1. VLfeat download
    VLfeat download
    insert image description here

  2. Select the corresponding file and move it into the current project folder
    insert image description here


  3. Find the direct installation path of the [sift.py] file in the corresponding directory according to whether your computer is directly installed with Python or Anaconda : python\Lib\site-packages\PCV\localdescriptors
    Anaconda installation path :Anaconda\Lib\site-packages\PCV\localdescriptors

  4. Open sift.py, modify the path
    Open the [sift.py] file, global cmmd , change the path in the quotation marks pointed by the arrow to the path of [sift.exe] in your project
    Note: if you use "\" in the path, you need Add "r" at the front end, use ''/'' or "\" is not required to
    insert image description here
    insert image description here
    successfully generate the sift file

cmmd = str(r"D:/LearningData/imageMosaic/panoramic-image/sift.exe "+imagename+" --output="+resultname+ " "+params)

4.4 Pictures are too different

ValueError: did not meet fit acceptance criteria

Multi-image stitching has relatively high requirements for images, and the stitching effect is poor if the difference is large or too small (almost the same). And if the shooting location changes too much, the matching value will be 0

5. Splicing result

5.1 Stitching of two images

  1. original image
    insert image description here
    insert image description here

  2. get feature
    insert image description here

  3. stitching result
    insert image description here

5.2 Stitching of two images from different angles

  1. original image
    insert image description here
    insert image description here

  2. feature
    insert image description here

  3. Results
    insert image description here
    There are certain differences in depth of field, but image stitching from different angles can still be stitched

5.3 Stitching of three pictures

  1. original image
    insert image description here
    insert image description here
    insert image description here
    insert image description here

  2. feature
    insert image description here
    insert image description here

  3. stitching result
    insert image description here

5.4 Four-image stitching

  1. original image
    insert image description here
    insert image description here
    insert image description here
    insert image description here

  2. feature map
    insert image description here
    insert image description here
    insert image description here

  3. result
    insert image description here

For the splicing of multiple pictures, there is still some distortion

5.5 Stitching of five pictures

  1. Original picture
    insert image description here
    insert image description here
    insert image description here
    insert image description here
    insert image description here
    insert image description here
    order from right to left

  2. feature
    insert image description here
    insert image description here
    insert image description here
    insert image description here

  3. result
    insert image description here

6. Combing the splicing process

  1. Read the image file under the corresponding path, and save the extracted features in the temporary file tmp.pgm
    insert image description here

  2. The feature is converted to the corresponding feature matrix, which is saved in the .sift file
    insert image description here

  3. Match the features of the adjacent two images respectively, and visualize the feature matching

  4. Find the corresponding change matrix through the encapsulated PCV method and the matched features, so as to perform transformation, distortion, and fusion. The
    corresponding transformation matrix is ​​as follows
    insert image description here

  5. Visualize stitching results
    insert image description here

appendix

1. Change address of PCV

Based on the fact that both problems 4 and 5 are caused by the python version, I uploaded the modified code to the
PCV code modified by myself on github

2. Two image splicing codes

from pylab import *
from numpy import *
from PIL import Image

# If you have PCV installed, these imports should work
from PCV.geometry import homography, warp
from PCV.localdescriptors import sift

"""
This is the panorama example from section 3.3.
"""

# set paths to data folder
# imname使我们要拼接的原图
# featname是sift文件，这个文件是需要根据原图进行生成的
# 需要根据自己的图像地址和图像数量修改地址和循环次数
# featname = ['./images5/'+str(i+1)+'.sift' for i in range(2)] 
# imname = ['./images5/'+str(i+1)+'.jpg' for i in range(2)]
featname = ['./test/1027-'+str(i+1)+'.sift' for i in range(2)] 
imname = ['./test/1027-'+str(i+1)+'.jpg' for i in range(2)]

# extract features and match
l = {
    
    }
d = {
    
    }
for i in range(2): 
    sift.process_image(imname[i],featname[i])
    l[i],d[i] = sift.read_features_from_file(featname[i])

matches = {
    
    }
for i in range(1):
    matches[i] = sift.match(d[i+1],d[i])

# visualize the matches (Figure 3-11 in the book)
for i in range(1):
    im1 = array(Image.open(imname[i]))
    im2 = array(Image.open(imname[i+1]))
    figure()
    sift.plot_matches(im2,im1,l[i+1],l[i],matches[i],show_below=True)


# function to convert the matches to hom. points
def convert_points(j):
    ndx = matches[j].nonzero()[0]
    fp = homography.make_homog(l[j+1][ndx,:2].T) 
    ndx2 = [int(matches[j][i]) for i in ndx]
    tp = homography.make_homog(l[j][ndx2,:2].T) 
    
    # switch x and y - TODO this should move elsewhere
    fp = vstack([fp[1],fp[0],fp[2]])
    tp = vstack([tp[1],tp[0],tp[2]])
    return fp,tp


# estimate the homographies
model = homography.RansacModel() 
# 此代码段为2图图像拼接，若需要多幅图，只需将其中的注释部分取消即可，图像顺序为自右向左。
fp,tp = convert_points(0)
H_01 = homography.H_from_ransac(fp,tp,model)[0] #im 0 to 1

#fp,tp = convert_points(1)
#H_12 = homography.H_from_ransac(fp,tp,model)[0] #im 1 to 2 

#tp,fp = convert_points(2) #NB: reverse order
#H_32 = homography.H_from_ransac(fp,tp,model)[0] #im 3 to 2 

#tp,fp = convert_points(3) #NB: reverse order
#H_43 = homography.H_from_ransac(fp,tp,model)[0] #im 4 to 3    


# warp the images
delta = 2000 # for padding and translation

im1 = array(Image.open(imname[0]), "uint8")
im2 = array(Image.open(imname[1]), "uint8")
im_12 = warp.panorama(H_01,im1,im2,delta,delta)
#im1 = array(Image.open(imname[0]), "f")
#im_02 = warp.panorama(dot(H_12,H_01),im1,im_12,delta,delta)

#im1 = array(Image.open(imname[3]), "f")
#im_32 = warp.panorama(H_32,im1,im_02,delta,delta)

#im1 = array(Image.open(imname[4]), "f")
#im_42 = warp.panorama(dot(H_32,H_43),im1,im_32,delta,2*delta)


figure()
imshow(array(im_12, "uint8"))
axis('off')
savefig("example1.png",dpi=300)
show()

3. Three images splicing code

# 博客方法（三张图）
from pylab import *
from numpy import *
from PIL import Image
 
# If you have PCV installed, these imports should work
from PCV.geometry import homography, warp
from PCV.localdescriptors import sift
np.seterr(invalid='ignore') # 忽略部分警告
"""
This is the panorama example from section 3.3.
"""
 
# 设置数据文件夹的路径
featname = ['test/1027-'+str(i+1)+'.sift' for i in range(3)] 
imname = ['test/1027-'+str(i+1)+'.jpg' for i in range(3)]
 
# 提取特征并匹配使用sift算法
l = {
    
    }
d = {
    
    }
for i in range(3):
    sift.process_image(imname[i], featname[i]) # 处理图像并将结果保存到文件中tmp.pgm，进而保存到.sift文件中
    # feature locations, descriptors要素位置，描述符
    l[i], d[i] = sift.read_features_from_file(featname[i]) # 读取特征属性并以矩阵形式返回
# 特征间两两匹配
matches = {
    
    }
for i in range(2):
    matches[i] = sift.match(d[i + 1], d[i])
 
# 可视化匹配
for i in range(2):
    im1 = array(Image.open(imname[i]))
    im2 = array(Image.open(imname[i + 1]))
    figure()
    # im1、im2（图像作为数组）、locs1、locs2（特征位置）,matchscores（作为“match”的输出），show_below（如果下面应该显示图像）
    sift.plot_matches(im2, im1, l[i + 1], l[i], matches[i], show_below=True)
 
 
# 将匹配转换成齐次坐标点的函数
def convert_points(j):
    ndx = matches[j].nonzero()[0]
    fp = homography.make_homog(l[j + 1][ndx, :2].T)
    ndx2 = [int(matches[j][i]) for i in ndx]
    tp = homography.make_homog(l[j][ndx2, :2].T)
 
    # switch x and y - TODO this should move elsewhere
    fp = vstack([fp[1], fp[0], fp[2]])
    tp = vstack([tp[1], tp[0], tp[2]])
    return fp, tp
 
 
# 估计单应性矩阵
model = homography.RansacModel()
# 博客方法 
fp, tp = convert_points(1)
H_12 = homography.H_from_ransac(fp, tp, model)[0]  # im 1 to 2
print(H_12, 'H_12')
 
fp, tp = convert_points(0)
H_01 = homography.H_from_ransac(fp, tp, model)[0]  # im 0 to 1
print(H_01, 'H_01')
 
# tp, fp = convert_points(2)  # NB: reverse order
# H_32 = homography.H_from_ransac(fp, tp, model)[0]  # im 3 to 2
 
# tp, fp = convert_points(3)  # NB: reverse order
# H_43 = homography.H_from_ransac(fp, tp, model)[0]  # im 4 to 3


 
# 扭曲图像
delta = 1500  # for padding and translation用于填充和平移

# 博客方法
im1 = array(Image.open(imname[1]), "uint8")
im2 = array(Image.open(imname[2]), "uint8")
im_12 = warp.panorama(H_12, im1, im2, delta, delta)

im1 = array(Image.open(imname[0]), "f")
im_02 = warp.panorama(dot(H_12, H_01), im1, im_12, delta, delta)

# im1 = array(Image.open(imname[3]), "f")
# im_32 = warp.panorama(H_32, im1, im_02, delta, delta)

# im1 = array(Image.open(imname[4]), "f")
# im_42 = warp.panorama(dot(H_32, H_43), im1, im_32, delta, 2 * delta)


figure()
imshow(array(im_02, "uint8"))
axis('off')
show()

4. Four image splicing code

from pylab import *
from numpy import *
from PIL import Image
 
# If you have PCV installed, these imports should work
from PCV.geometry import homography, warp
from PCV.localdescriptors import sift
np.seterr(invalid='ignore') # 忽略部分警告
"""
This is the panorama example from section 3.3.
"""
 
# 设置数据文件夹的路径
featname = ['test/1027-'+str(i+1)+'.sift' for i in range(4)] 
imname = ['test/1027-'+str(i+1)+'.jpg' for i in range(4)]
 
# 提取特征并匹配使用sift算法
l = {
    
    }
d = {
    
    }
for i in range(4):
    sift.process_image(imname[i], featname[i]) # 处理图像并将结果保存到文件中tmp.pgm，进而保存到.sift文件中
    # feature locations, descriptors要素位置，描述符
    l[i], d[i] = sift.read_features_from_file(featname[i]) # 读取特征属性并以矩阵形式返回
 
matches = {
    
    }
for i in range(3):
    matches[i] = sift.match(d[i + 1], d[i])
 
# 可视化匹配
for i in range(3):
    im1 = array(Image.open(imname[i]))
    im2 = array(Image.open(imname[i + 1]))
    figure()
    # im1、im2（图像作为数组）、locs1、locs2（特征位置）,matchscores（作为“match”的输出），show_below（如果下面应该显示图像）
    sift.plot_matches(im2, im1, l[i + 1], l[i], matches[i], show_below=True)
 
 
# 将匹配转换成齐次坐标点的函数
def convert_points(j):
    ndx = matches[j].nonzero()[0]
    fp = homography.make_homog(l[j + 1][ndx, :2].T)
    ndx2 = [int(matches[j][i]) for i in ndx]
    tp = homography.make_homog(l[j][ndx2, :2].T)
 
    # switch x and y - TODO this should move elsewhere
    fp = vstack([fp[1], fp[0], fp[2]])
    tp = vstack([tp[1], tp[0], tp[2]])
    return fp, tp
 
 
# 估计单应性矩阵
model = homography.RansacModel()
# 博客方法 
fp, tp = convert_points(1)
H_12 = homography.H_from_ransac(fp, tp, model)[0]  # im 1 to 2
 
fp, tp = convert_points(0)
H_01 = homography.H_from_ransac(fp, tp, model)[0]  # im 0 to 1
 
tp, fp = convert_points(2)  # NB: reverse order
H_32 = homography.H_from_ransac(fp, tp, model)[0]  # im 3 to 2
 
# tp, fp = convert_points(3)  # NB: reverse order
# H_43 = homography.H_from_ransac(fp, tp, model)[0]  # im 4 to 3

 
# 扭曲图像
delta = 2000  # for padding and translation用于填充和平移

# 博客方法
im1 = array(Image.open(imname[1]), "uint8")
im2 = array(Image.open(imname[2]), "uint8")
im_12 = warp.panorama(H_12, im1, im2, delta, delta)
 
im1 = array(Image.open(imname[0]), "f")
im_02 = warp.panorama(dot(H_12, H_01), im1, im_12, delta, delta)
 
im1 = array(Image.open(imname[3]), "f")
im_32 = warp.panorama(H_32, im1, im_02, delta, delta)
 
# im1 = array(Image.open(imname[4]), "f")
# im_42 = warp.panorama(dot(H_32, H_43), im1, im_32, delta, 2 * delta)



figure()
imshow(array(im_32, "uint8"))
axis('off')
show()

5. Five image splicing codes

# -*- codeing =utf-8 -*-
# @Time : 2021/4/20 11:00
# @Author : ArLin
# @File : demo1.py
# @Software: PyCharm
from pylab import *
from numpy import *
from PIL import Image

# If you have PCV installed, these imports should work
from PCV.geometry import homography, warp
from PCV.localdescriptors import sift
np.seterr(invalid='ignore')
"""
This is the panorama example from section 3.3.
"""

# 设置数据文件夹的路径
featname = ['./test/1027-'+str(i+1)+'.sift' for i in range(5)] 
imname = ['./test/1027-'+str(i+1)+'.jpg' for i in range(5)]

# 提取特征并匹配使用sift算法
l = {
    
    }
d = {
    
    }
for i in range(5):
    sift.process_image(imname[i], featname[i])
    l[i], d[i] = sift.read_features_from_file(featname[i])

matches = {
    
    }
for i in range(4):
    matches[i] = sift.match(d[i + 1], d[i])

# 可视化匹配
for i in range(4):
    im1 = array(Image.open(imname[i]))
    im2 = array(Image.open(imname[i + 1]))
    figure()
    sift.plot_matches(im2, im1, l[i + 1], l[i], matches[i], show_below=True)


# 将匹配转换成齐次坐标点的函数
def convert_points(j):
    ndx = matches[j].nonzero()[0]
    fp = homography.make_homog(l[j + 1][ndx, :2].T)
    ndx2 = [int(matches[j][i]) for i in ndx]
    tp = homography.make_homog(l[j][ndx2, :2].T)

    # switch x and y - TODO this should move elsewhere
    fp = vstack([fp[1], fp[0], fp[2]])
    tp = vstack([tp[1], tp[0], tp[2]])
    return fp, tp


# 估计单应性矩阵
model = homography.RansacModel()

fp, tp = convert_points(1)
H_12 = homography.H_from_ransac(fp, tp, model)[0]  # im 1 to 2

fp, tp = convert_points(0)
H_01 = homography.H_from_ransac(fp, tp, model)[0]  # im 0 to 1

tp, fp = convert_points(2)  # NB: reverse order
H_32 = homography.H_from_ransac(fp, tp, model)[0]  # im 3 to 2

tp, fp = convert_points(3)  # NB: reverse order
H_43 = homography.H_from_ransac(fp, tp, model)[0]  # im 4 to 3

# 扭曲图像
delta = 2000  # for padding and translation用于填充和平移

im1 = array(Image.open(imname[1]), "uint8")
im2 = array(Image.open(imname[2]), "uint8")
im_12 = warp.panorama(H_12, im1, im2, delta, delta)

im1 = array(Image.open(imname[0]), "f")
im_02 = warp.panorama(dot(H_12, H_01), im1, im_12, delta, delta)

im1 = array(Image.open(imname[3]), "f")
im_32 = warp.panorama(H_32, im1, im_02, delta, delta)

im1 = array(Image.open(imname[4]), "f")
im_42 = warp.panorama(dot(H_32, H_43), im1, im_32, delta, 2 * delta)

figure()
imshow(array(im_42, "uint8"))
axis('off')
show()

Guess you like

Origin blog.csdn.net/m0_47146037/article/details/126855241
Recommended
Ranking
Empire cms smart tag calls four first-level recommended articles, starting from the fourth article
Linux environment installation and configuration Elasticsearch7.17
Big Data processing architecture and Lambda Kappa architecture
Explore the top of the AI large model platform - Wenxin Qianfan
Beijing car PK10 lucky airship Guanya size and value of the odd and even tips
W3B x Sui Hacker House｜In-depth understanding of Sui and Move language
Know almost Ko Chan: Chinese what any decent open source software products? (Finishing from my original answer)
Comprehensively improve AD domain security authentication | Zhuyun IDaaS
Android Update Engine Analysis (24) What happened when making the downgrade package?
Spark Architecture and Operating Mechanism (1) - System Architecture
Daily
More
2024-05-07(34)
2024-05-06(6)
2024-05-05(0)
2024-05-04(18)
2024-05-03(8)
2024-05-02(0)
2024-05-01(4)
2024-04-30(36)
2024-04-29(5)
2024-04-28(12)

Code World

© 2018 codetd.com