multi-view stereo document

multi view stereo document

This document summarizes the author's use Pay Per View Stereo View Multi ( MVS method) to the camera rectify, depth map generation process, hoping to give the students after some help.

About MVS

Multi View Stereo, is for a multi-view three-dimensional reconstruction of the general term for a range of methods, in short, is the use of multiple pictures of multiple cameras to reconstruct a picture of the scene.

Multi view stereo learning process following some classic resource, you can click on the following link to download the form.

content	description
multiview stereo a tutorial	mutli view stereo classic tutorial is highly recommended reading.
pixelwise... MVS	colmap team to write papers
structure from motion	Understand the structure from motion very good material, a chapter of a book about sfm

Complete multi view stereo pipeline will have the following steps

• input images
• structure from motion(SFM) => camera parameters, sparse point cloud
• Muti view stereo(MVS) => depth map, dense point cloud
• Surface reconstruction(SR) => poisson or delauny reconstruction, mesh
• texture mapping(TM）=> get mesh with texture

sfm process can be represented at this picture

MVS tool chain

Several open-source project summary

As previously described, MVS pipeline has four main steps, SFM, MVS, SR (TM) and, respectively, these steps can refer to the following open-source tool, click on the link to go directly to the respective engineering

• SFM:
  • visual sfm very robust in sfm tools, can be downloaded directly under the window-packaged binary, under mac and Linux If you manually compile complex (very recommended manual translation, the author on this issue the card for a few days, under a compromise with windows also a good choice)
  • open MVG I never ran this project, the effect is unknown, then you can try
  • colmap interface is simple and easy to use, but can only be used in dense reconstruction windows download good cuda, recommended

• MVS

  • PMVS-CMVS classic MVS tool, first with CMVS classification of the input image, then PMVS reconstruction, integration, very suitable for large-scale data
  • colmap convenient generating a depth map
  • openMvs I never ran this project, visually very good run, a lot of people running in the process encountered a problem.

• SR && TM

  • Default general visual sfm, colmap comes with the SR, TM tool, you can use the default tool

  • meshlab provides mesh simplify, smooth, SR-related functions such as mesh, triangular mesh for good results. Meshlab download the installation package is recommended for beginners to use, if you want deep dive details, you can download vcglib (mesh lab developed based on this lib), compile the source code, vcglib with the idea of a lot of meta-programming, or a lot of fun, but the process is more complex, Shenru.

colmap Tutorial

In this chapter, the author describes colmap use of tutorials.

installation

colmap recommend downloading pre built binary, it is very stable, if you want to use dense reconstruction, cuda install yourself.

Click mac system macos

windows system Please click on the windows

Point directly after run, the installation is complete.

structure from motion

FIG colmap process is as follows

The first step is sfm, the need to input all of the pictures into a folder, and then click

Feature extraction and matching the results of extraction may be visualized in the database management.

After clicking start reconstruction sparse bundle adjustment and reconstruction.

It is worth mentioning that there is good or bad evaluation reprojection error of camera parameters in the parameter extras-> model statistic, the better result is generally about 0.5.

export model and rectify

After carrying out the sparse reconstruction, the model can be exported to txt file.

includes

• Camera parameters
• Feature points
• 3d point cloud information

By scripts read the corresponding parameter in the script provided.

If further rectify, the input parameters may be the opencv stereoRectify()function, then initUndistortRecifyMap(), the last used remap()for rectify.

dense reconstruction and depth map

Reconstruction according undistoriont, stereo, fusion, poisson process in the screen below

dense reconstruction of the depth chart From the above chart, for the texture is not very rich in pictures, it is recommended to increase windows_radius and reduce filter_min_ncc to improve the accuracy of the depth map.

PMVS Tutorial

PMVS can refer to this blog installation https://blog.csdn.net/moneyhoney123/article/details/78454837

Note that a pit point is that if the picture is too small, will be reported pmvs2 crash not enough memory PMVS need to modify the configuration file nv.ini, the minImageNum to 2