[3D Reconstruction] [Deep Learning] Pytorch implementation of instant-nsr-pl code--train your own data set

[3D Reconstruction] [Deep Learning] Pytorch implementation of instant-nsr-pl code – train your own data set

Instant-NGP based neural surface reconstruction: This repository contains concise and scalable implementations of NeRF and NeuS for neural surface reconstruction based on Instant-NGP and Pytorch-Lightning frameworks, intended for research projects based on NeRF or NeuS Provide efficient and customizable templates.

---

Preface

Before analyzing the NeuS network in detail, the first task is to build the operating environment required by instant-nsr-pl [ reference tutorial under win10 ], and complete the training and testing of the model. Only then will it make sense to carry out subsequent work.
This blog post explains the process of training your own data set with the instant-nsr-pl network. The detailed code will be explained in subsequent blog posts.

The blogger has analyzed the codes of each functional module in detail in different blog posts. Click [Reference Tutorial under Win10], and the directory link of the blog post is placed in the preface.

---

Make a dataset

Regarding the detailed process of making a personal data set, those who are interested can refer to another blog post by the blogger [ Create your own NeRF (LLFF format) data set based on COLMAP ]. The instant-nsr-pl source code has its own production method:

1. Download the COLMAP software , select the appropriate version to download and unzip it, and put all the files in the bin and lib folders in the same directory.
   

2. Configure the environment variable of colmap and enter "colmap" in cmd to verify whether it takes effect.
   

3. To convert the data format, there is images in the giraffe directory to place all pictures, and sparse\0 (created by yourself) will save the results output by colmap.
   

   activate instant-nsr-pl
   python scripts/imgs2poses.py ./load/unbounded360/giraffe
   

The data format conversion is completed here:

---

Train your own data set

Let me emphasize that you should pay attention to the memory. If it is not enough, expand the virtual memory.

# nerf 
python launch.py --config configs/nerf-colmap.yaml --gpu 0 --train dataset.scene=giraffe tag=giraffe
# neus带有mask
python launch.py --config configs/neus-colmap.yaml --gpu 0 --train dataset.scene=giraffe tag=giraffe 
# neus没有mask
python launch.py --config configs/neus-colmap.yaml --gpu 0 --train dataset.scene=giraffe tag=giraffe dataset.use_mask=false

There is a huge BUG in the code of the source code author here, and it may also be my own fault. Anyway, the training of my own data set is a mess! ! ! ! If anyone knows, please tell me how to deal with it in the comment area. I will update if there is a solution later! ! !

---

Summarize

Introduce the process of instant-nsr-pl training your own data set as simply and in detail as possible. Later, I will explain the principles and code of instant-nsr-pl based on the knowledge I have learned and my personal understanding.