Instant NeRF - Study&Debug
This machine is configured with Y9000P RTX3060 Win11
1. Git
-
Download and install normally, start Git Bash
-
set username
$ git config --global user.name "**"
$ git config --global user.email "–*******@gmail.com"
$ git config --global --list
user.name=**
user.email=<U+0096>********@gmail.com
Since the above mailbox is copied from the browser, there is a • in front , which is <U+0096>, but it is not visible on the terminal (something copied by the browser often has this thing). The solution is to re-enter the command manually
$ git config --global user.email "*********@gmail.com"
- Link to Github, see the online tutorial for details, the password here is too lazy to remember, it is empty
$ ssh-keygen -t rsa -C "*******@gmail.com"
Generating public/private rsa key pair.
Enter file in which to save the key (/c/Users/****/.ssh/id_rsa):
Created directory '/c/Users/****/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /c/Users/****.ssh/id_rsa
Your public key has been saved in /c/Users/****/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:****************** ****@gmail.com
The key's randomart image is:
+---[RSA 3072]----+
| ... o.=. . |
| . ****. |
| o . .o.*o. ..|
|. . .. =o . |
|E***** |
|=.O + |
|oB.----- |
|= =.B |
|.+ *. |
+----[SHA256]-----+
- Try to ping the Github remote warehouse, success
$ ssh [email protected]
The authenticity of host 'github.com (20.205.243.166)' can't be established.
ED25519 key fingerprint is SHA256:+DiY3wvvV6TuJJhbpZisF/zLDA0zPMSvHdkr4UvCOqU.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'github.com' (ED25519) to the list of known hosts.
PTY allocation request failed on channel 0
Hi **********! You've successfully authenticated, but GitHub does not provide shell access.
Connection to github.com closed.
- git clone reports an error
$ git clone --recursive https://github.com/nvlabs/instant-ngp
- Connection was reset, errno 10054
fatal: unable to access 'https://github.com/nvlabs/instant-ngp/': OpenSSL SSL_read: Connection was reset, errno 10054
solution:
$ git config --global http.sslVerify false
- port 443 after 21083 ms: Timed out
fatal: unable to access 'https://github.com/nvlabs/instant-ngp/': Failed to connect to github.com port 443 after 21083 ms: Timed out
Proxy problem, add the port number set in Clash to config
solution:
$ git config --global http.proxy http://127.0.0.1:7890
$ git config --global https.proxy https://127.0.0.1:7890
2. Cmake
Execute in the developer command prompt window (developer command prompt directly use this English search)
$ cd instant-ngp
instant-ngp$ cmake . -B build
The output is as follows:
D:\Project\CmakeProject\instant-ngp>cmake . -B build
-- Building for: Visual Studio 16 2019
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.22000.
-- The C compiler identification is MSVC 19.29.30145.0
-- The CXX compiler identification is MSVC 19.29.30145.0
-- The CUDA compiler identification is NVIDIA 11.1.105
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: D:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: D:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: D:/Program Files (x86)/NVIDIA/NVIDIA GPU Computing Toolkit/CUDA/v11.1/bin/nvcc.exe - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Targeting GPU architectures: 86
-- Module support is disabled.
-- Version: 9.0.0
-- Build type:
-- CXX_STANDARD: 14
-- Required features: cxx_variadic_templates
-- Could NOT find Vulkan (missing: Vulkan_LIBRARY Vulkan_INCLUDE_DIR) (found version "")
CMake Warning at CMakeLists.txt:118 (message):
Vulkan was not found. Neural graphics primitives will still compile and
run correctly, but DLSS will not be supported.
-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Found Threads: TRUE
-- Using Win32 for window creation
-- Found OpenMP_C: -openmp (found version "2.0")
-- Found OpenMP_CXX: -openmp (found version "2.0")
-- Found OpenMP: TRUE (found version "2.0")
-- !!! Warning OptiX_INSTALL_DIR not set in environment. using default
-- OptiX_INSTALL_DIR value: C:\ProgramData\NVIDIA Corporation\OptiX SDK 7.5.0
-- OptiX headers (optix.h and friends) not found.
CMake Warning at CMakeLists.txt:194 (message):
OptiX was not found. Neural graphics primitives will still compile and run
correctly. However, SDF training in 'raystab' and 'pathescape' modes will
be significantly slower.
-- Found Python: D:/ProgramData/Anaconda3/python.exe (found suitable version "3.9.12", minimum required is "3.7") found components: Interpreter Development Development.Module Development.Embed
-- pybind11 v2.7.1
CMake Warning (dev) at D:/Program Files (x86)/CMake/share/cmake-3.23/Modules/CMakeDependentOption.cmake:89 (message):
Policy CMP0127 is not set: cmake_dependent_option() supports full Condition
Syntax. Run "cmake --help-policy CMP0127" for policy details. Use the
cmake_policy command to set the policy and suppress this warning.
Call Stack (most recent call first):
dependencies/pybind11/CMakeLists.txt:98 (cmake_dependent_option)
This warning is for project developers. Use -Wno-dev to suppress it.
-- Performing Test HAS_MSVC_GL_LTCG
-- Performing Test HAS_MSVC_GL_LTCG - Success
-- Configuring done
-- Generating done
-- Build files have been written to: D:/Project/CmakeProject/instant-ngp/build
Some things are missing, try installing an OptiX, I see most of the tutorials have installed this stuff
The location of the installation is D:\Program Files (x86)\NVIDIA\NVIDIA Corporation\OptiX SDK 7.5.0
that you need to put it in the system variable (N is the variable name, V is the variable value)
N: OptiX_INSTALL_DIR
V: D:\Program Files (x86)\NVIDIA\NVIDIA Corporation\OptiX SDK 7.5.0
Adding it alone is not enough, you need to update the system variables, set PATH=C:
, close the shell and open it again echo %PATH%
, restart cmake, the output is as follows
D:\Project\CmakeProject\instant-ngp>cmake . -B build
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.22000.
-- Targeting GPU architectures: 86
-- Module support is disabled.
-- Version: 9.0.0
-- Build type:
-- CXX_STANDARD: 14
-- Required features: cxx_variadic_templates
-- Could NOT find Vulkan (missing: Vulkan_LIBRARY Vulkan_INCLUDE_DIR) (found version "")
CMake Warning at CMakeLists.txt:118 (message):
Vulkan was not found. Neural graphics primitives will still compile and
run correctly, but DLSS will not be supported.
-- Using Win32 for window creation
-- OptiX_INSTALL_DIR value: D:\Program Files (x86)\NVIDIA\NVIDIA Corporation\OptiX SDK 7.5.0
-- pybind11 v2.7.1
CMake Warning (dev) at D:/Program Files (x86)/CMake/share/cmake-3.23/Modules/CMakeDependentOption.cmake:89 (message):
Policy CMP0127 is not set: cmake_dependent_option() supports full Condition
Syntax. Run "cmake --help-policy CMP0127" for policy details. Use the
cmake_policy command to set the policy and suppress this warning.
Call Stack (most recent call first):
dependencies/pybind11/CMakeLists.txt:98 (cmake_dependent_option)
This warning is for project developers. Use -Wno-dev to suppress it.
-- Configuring done
-- Generating done
-- Build files have been written to: D:/Project/CmakeProject/instant-ngp/build
OptiX is finished, but there are still many problems, first look at Vulkan
-
Also download and install, according to its default installation directory, the actual installation directory is adjusted to
D:\Program Files (x86)\NVIDIA\VulkanSDK\1.3.224.1
-
Also add environment variables, it is estimated that the cmake prompt requires two, respectively
N: Vulkan_LIBRARY V: D:\Program Files (x86)\NVIDIA\VulkanSDK\1.3.224.1\Lib N: Vulkan_INCLUDE_DIR V: D:\Program Files (x86)\NVIDIA\VulkanSDK\1.3.224.1\Include
-
Also update environment variables (see above)
-
Restart Cmake, the output is as follows
D:\Project\CmakeProject\instant-ngp>cmake . -B build -- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.22000. -- Targeting GPU architectures: 86 -- Module support is disabled. -- Version: 9.0.0 -- Build type: -- CXX_STANDARD: 14 -- Required features: cxx_variadic_templates -- Found Vulkan: D:/Program Files (x86)/NVIDIA/VulkanSDK/1.3.224.1/Lib/vulkan-1.lib (found version "1.3.224") -- Using Win32 for window creation -- OptiX_INSTALL_DIR value: D:\Program Files (x86)\NVIDIA\NVIDIA Corporation\OptiX SDK 7.5.0 -- pybind11 v2.7.1 CMake Warning (dev) at D:/Program Files (x86)/CMake/share/cmake-3.23/Modules/CMakeDependentOption.cmake:89 (message): Policy CMP0127 is not set: cmake_dependent_option() supports full Condition Syntax. Run "cmake --help-policy CMP0127" for policy details. Use the cmake_policy command to set the policy and suppress this warning. Call Stack (most recent call first): dependencies/pybind11/CMakeLists.txt:98 (cmake_dependent_option) This warning is for project developers. Use -Wno-dev to suppress it. -- Configuring done -- Generating done -- Build files have been written to: D:/Project/CmakeProject/instant-ngp/build
It seems that only cmake related issues are left
Execute according to the prompt
cmake --help-policy CMP0127
D:\Project\CmakeProject\instant-ngp>cmake --help-policy CMP0127 CMP0127 ------- .. versionadded:: 3.22 ``cmake_dependent_option()`` supports full :ref:`Condition Syntax`. The ``<depends>`` parameter accepts a :ref:`semicolon-separated list <CMake Language Lists>` of conditions. CMake 3.21 and lower evaluates each ``condition`` as ``if(${ condition})``, which does not properly handle conditions with nested paren groups. CMake 3.22 and above instead prefer to evaluate each ``condition`` as ``if(<condition>)``, where ``<condition>`` is re-parsed as if literally written in a call to ``if()``. This allows expressions like:: "A AND (B OR C)" but requires expressions like:: "FOO MATCHES (UPPER|lower)" to be re-written as:: "FOO MATCHES \"(UPPER|lower)\"" Policy ``CMP0127`` provides compatibility for projects that have not been updated to expect the new behavior. This policy was introduced in CMake version 3.22. CMake version 3.23.2 warns when the policy is not set and uses ``OLD`` behavior. Use the ``cmake_policy()`` command to set it to ``OLD`` or ``NEW`` explicitly. .. note:: The ``OLD`` behavior of a policy is ``deprecated by definition`` and may be removed in a future version of CMake.
It’s too abstract, I don’t understand it at first glance, for Google programming, combined with cmake output, try n times, and finally refer to this blog to solve it. The essence is that it must be added before the error line, good guy, good guy
According to the cmake prompt, the line where the error is reported dependencies/pybind11/CMakeLists.txt:98 (cmake_dependent_option)
, find this CMakeLists.txt and add the following three lines of code on line 97
if(POLICY CMP0127)
cmake_policy(SET CMP0127 NEW)
endif()
cmake_dependent_option(PYBIND11_FINDPYTHON "Force new FindPython" OFF
"NOT CMAKE_VERSION VERSION_LESS 3.12" OFF)
restart cmake
D:\Project\CmakeProject\instant-ngp>cmake . -B build
-- Selecting Windows SDK version 10.0.19041.0 to target Windows 10.0.22000.
-- Targeting GPU architectures: 86
-- Module support is disabled.
-- Version: 9.0.0
-- Build type:
-- CXX_STANDARD: 14
-- Required features: cxx_variadic_templates
-- Using Win32 for window creation
-- OptiX_INSTALL_DIR value: D:\Program Files (x86)\NVIDIA\NVIDIA Corporation\OptiX SDK 7.5.0
-- pybind11 v2.7.1
-- Configuring done
-- Generating done
-- Build files have been written to: D:/Project/CmakeProject/instant-ngp/build
Cmake succeeds, no error is reported, it just does not make the Module support is disabled on line 4. What does it mean? Confirm
It seems that no one has asked this question, and the sample code posted on Github also has this sentence, let’s ignore it
Next is the next cmake command, which is also executed in the developer command prompt
instant-ngp$ cmake --build build --config RelWithDebInfo -j
A bunch of errors, too much output, put a few error reports to see
D:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\include\xutility(1260): error :
expected a "(" [D:\Project\CmakeProject\instant-ngp\build\optix_program.vcxproj]
detected during instantiation of "void std::_Adl_verify_range(const _Iter &, const _Sentinel &) [with _Iter
=const char *, _Sentinel=const char *]"
D:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\include\xlocale(1971): here
D:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\include\xutility(1261): error :
identifier "_Verify_range" is undefined [D:\Project\CmakeProject\instant-ngp\build\optix_program.vcxproj]
detected during instantiation of "void std::_Adl_verify_range(const _Iter &, const _Sentinel &) [with _Iter
=const char *, _Sentinel=const char *]"
D:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\include\xlocale(1971): here
D:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\include\xutility(1261): error :
identifier "_Verify_range" is undefined [D:\Project\CmakeProject\instant-ngp\build\optix_program.vcxproj]
detected during instantiation of "void std::_Adl_verify_range(const _Iter &, const _Sentinel &) [with _Iter
=const char *, _Sentinel=const char *]"
D:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\include\xlocale(1971): here
D:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\BuildCustomizations\CUDA 11.1.t
argets(785,9): error MSB3721: 命令“"D:\Program Files (x86)\NVIDIA\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc.exe"
--use-local-env -ccbin "D:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\bin\Hos
tX64\x64" -x cu -I"D:\Program Files (x86)\NVIDIA\VulkanSDK\1.3.224.1\Include" -I"D:\Project\CmakeProject\instant-ngp\
dependencies\dlss\include" -I"D:\Project\CmakeProject\instant-ngp\dependencies\glfw\include" -I"D:\Project\CmakeProject
\instant-ngp\dependencies\imgui" -I"D:\Project\CmakeProject\instant-ngp\dependencies\gl3w" -I"D:\Project\CmakeProject\i
nstant-ngp\dependencies" -I"D:\Project\CmakeProject\instant-ngp\dependencies\eigen" -I"D:\Project\CmakeProject\instant-
ngp\dependencies\filesystem" -I"D:\Project\CmakeProject\instant-ngp\dependencies\nanovdb" -I"D:\Project\CmakeProject\in
stant-ngp\dependencies\tinylogger" -I"D:\Program Files (x86)\NVIDIA\NVIDIA Corporation\OptiX SDK 7.5.0\include" -I"D:\P
roject\CmakeProject\instant-ngp\include" -I"D:\Project\CmakeProject\instant-ngp\dependencies\tiny-cuda-nn\include" -I"D
:\Project\CmakeProject\instant-ngp\dependencies\tiny-cuda-nn\dependencies" -I"D:\Project\CmakeProject\instant-ngp\depen
dencies\tiny-cuda-nn\dependencies\cutlass\include" -I"D:\Project\CmakeProject\instant-ngp\dependencies\tiny-cuda-nn\dep
endencies\cutlass\tools\util\include" -I"D:\Project\CmakeProject\instant-ngp\dependencies\tiny-cuda-nn\dependencies\fmt
\include" -I"D:\Program Files (x86)\NVIDIA\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" --keep-dir x64\RelWithD
ebInfo -maxrregcount=0 --machine 64 -ptx -cudart shared --expt-relaxed-constexpr -std=c++14 -Xcompiler="/EHsc -Zi -Ob1
" -o optix_program.dir\RelWithDebInfo\raystab.ptx -D_WINDOWS -DNDEBUG -DNGP_VULKAN -DGLFW_INCLUDE_VULKAN -DNGP_GUI -DN
GP_OPTIX -D"NGP_VERSION=\"1.0dev\"" -DTCNN_MIN_GPU_ARCH=0 -D"CMAKE_INTDIR=\"RelWithDebInfo\"" -D_MBCS -D"CMAKE_INTDIR=\
"RelWithDebInfo\"" "D:\Project\CmakeProject\instant-ngp\src\optix\raystab.cu"”已退出,返回代码为 1。 [D:\Project\CmakeProject\ins
tant-ngp\build\optix_program.vcxproj]
D:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\include\xmemory(58): error : ex
pected a "(" [D:\Project\CmakeProject\instant-ngp\build\optix_program.vcxproj]
The keyword [D:\Project\CmakeProject\instant-ngp\build\optix_program.vcxproj] is probably related to it.
Finally found the reason , it has nothing to do with Optix, it is a problem with the CUDA version, so the next step is to install CUDA 11.6
After installing, restart cmake
cmake . -B build
cmake --build build --config RelWithDebInfo -j 16
Wait for three minutes, complete without error, too much output, only put the last few lines here
正在创建库 D:/Project/CmakeProject/instant-ngp/build/RelWithDebInfo/testbed.lib 和对象 D:/Project/CmakeProject/instant-ngp/
build/RelWithDebInfo/testbed.exp
testbed.vcxproj -> D:\Project\CmakeProject\instant-ngp\build\testbed.exe
python_api.cu
正在创建库 D:/Project/CmakeProject/instant-ngp/build/RelWithDebInfo/pyngp.lib 和对象 D:/Project/CmakeProject/instant-ngp/bu
ild/RelWithDebInfo/pyngp.exp
pyngp.vcxproj -> D:\Project\CmakeProject\instant-ngp\build\pyngp.cp39-win_amd64.pyd
Building Custom Rule D:/Project/CmakeProject/instant-ngp/CMakeLists.txt
3. instant-ngp
Then start using
Anaconda creates a virtual environment
conda create -n ngp python=3.9
conda activate ngp
pip install -r requirements.txt
sample data
I ran the data of the fox, and it feels okay.
instant-ngp$ ./build/testbed --scene data/nerf/fox
custom data
- Install Colmap
If you want to play it yourself, you can’t do without it. You can use it immediately after downloading, without the installation process. According to the usual practice, the software that can customize the installation path is moved to the directory D:\Program Files (x86)
. Add environment variables
N: Path
V: D:\Program Files (x86)\COLMAP\COLMAP-3.7-windows-cuda
-
Calculate pose
-
Put your custom image set in
data/<image_set_name>
-
transform.json
Get it from the following command. Insert your image path into<image/path>
python scripts/colmap2nerf.py --colmap_matcher exhaustive --run_colmap --aabb_scale 16 --images <image/path>
transform.json
will be generated in the root folder, drag and drop it into yourdata/<image_set_name>
folder
-
-
Follow the example data fox to reorganize the data structure and modify
transform.json
the image path
After moving the root directory to transform.json
the location, remember to modify the image path in it, and also follow the example data, just images/img.jpg
like
- start running
Try it with a photo you took
python scripts/colmap2nerf.py --colmap_matcher exhaustive --run_colmap --aabb_scale 16 --images data/earphone
Reorganize data, rewrite transform.json, run
instant-ngp$ ./build/testbed --scene data/nerf/earphone
Precautions
- The photos taken with the mobile phone should be taken horizontally or vertically, otherwise an error will be reported when estimating the pose
- A certain degree of overlap is required. For small objects, a dozen or dozens of sheets are enough
- If the pixel is too high, you can consider shrinking it by a factor of 1 or something before reprocessing, the code is as follows
#include<iostream>
#include<icecream.hpp>
#include <vector>
#include <io.h>
#include <fstream>
#include <string>
#include <opencv2/opencv.hpp>
#include<opencv2/highgui.hpp>
#include <opencv2/highgui/highgui_c.h>
using namespace std;
/************************************************************************/
/* 获取文件夹下所有文件名
输入:
path : 文件夹路径
exd : 所要获取的文件名后缀,如jpg、png等;如果希望获取所有
文件名, exd = ""
输出:
files : 获取的文件名列表
/************************************************************************/
void getFiles(string path, string exd, vector<string>& files)
{
//文件句柄
long long hFile = 0;
//文件信息
struct _finddata_t fileinfo;
string pathName, exdName;
if (0 != strcmp(exd.c_str(), ""))
{
exdName = "\\*." + exd;
}
else
{
exdName = "\\*";
}
if ((hFile = _findfirst(pathName.assign(path).append(exdName).c_str(), &fileinfo)) != -1)
{
do
{
//如果是文件夹中仍有文件夹,迭代之
//如果不是,加入列表
// 不推荐使用,硬要使用的话,需要修改else 里面的语句
/*if((fileinfo.attrib & _A_SUBDIR))
{
if(strcmp(fileinfo.name,".") != 0 && strcmp(fileinfo.name,"..") != 0)
getFiles( pathName.assign(path).append("\\").append(fileinfo.name), exd, files );
}
else */
{
if (strcmp(fileinfo.name, ".") != 0 && strcmp(fileinfo.name, "..") != 0)
//files.push_back(pathName.assign(path).append("\\").append(fileinfo.name)); // 要得到绝对目录使用该语句
//如果使用
files.push_back(fileinfo.name); // 只要得到文件名字使用该语句
}
} while (_findnext(hFile, &fileinfo) == 0);
_findclose(hFile);
}
}
int main()
{
string rootPath = "D:\\Project\\CmakeProject\\instant-ngp\\data\\nerf\\earphone_2";
vector<string> imgLists;
getFiles(rootPath, "jpg", imgLists);
IC(imgLists);
for (int i = 0; i < imgLists.size(); i++)
{
cv::Mat img = cv::imread(rootPath + "\\" + imgLists[i]);
cv::Mat dst;
cv::resize(img, dst, cv::Size(img.cols / 2, img.rows / 2), 0, 0, cv::INTER_AREA);
cv::imwrite(rootPath + "\\rsz_" + imgLists[i], dst);
}
return 0;
}
doubts
- It doesn't matter if you use sample data or your own data. There will be afterimages of photos near the camera pose
- It would be nice to have a tutorial for the GUI, but I still don’t quite understand the specific meaning of some interactive parameters
- The mesh effect is too bad to be used, the exported point cloud has a size of 100 million points (it is really hundreds of millions), and it cannot be opened