Article directory
0 Preface
In 2020, the emergence of NeRF caused an uproar and a large number of related works appeared. 3D gaussian is considered the current SOTA in the field of new perspective generation, and can perform real-time rendering; the maximum training speed is comparable to Instant, and the quality is similar; after increasing the number of iterations, the reconstruction quality can be significantly improved. With a training time of 51 minutes, The reconstruction effect can slightly exceed Mip-NeRF (48h). Who doesn’t want to learn this kind of good stuff, so let’s take the first step: configure the 3D gaussian environment.
My current environment is ubuntu 20.04+Quadro RTX 5000. This project has relatively high requirements for video memory. The official statement requires 24G of video memory. If the video memory is not enough, you can see the FAQ at the bottom of the github project homepage, which explains low-video memory solutions. In addition, the Viewer version currently seems to only support windows, ubuntu20.04 and 22.04.
There is a tutorial for installing in windows on the github homepageyoutue tutorial, which is very detailed.
Currently in the learning stage, simply use existing data sets for testing. After verifying that the environment is available, prepare to start reading the source code. Therefore, colmap has not been installed to process its own data, and the configuration of colmap will be supplemented later. Two new articles will be opened later to explain the paper and code.
1. cuda installation steps
1.1 Graphics card driver installation
Disable the system's own driver
If you do not disable this open source driver, conflicts may occur during subsequent installations. Use lsmod | grep nouveau to check the status of the driver. If there is output, it means that the driver is working, otherwise it means that it has been disabled.
The disabling method is as follows:
sudo gedit /etc/modprobe.d/blacklist.conf
Add at the end of the file
blacklist nouveau
options nouveau modeset=0
Enter in the terminalsudo update -initramfs -u
nvidia graphics card driver installation
The graphics card is hardware. If you want the graphics card to work properly, you must find the corresponding driver. Otherwise, various display problems will occur, such as the monitor not lighting up and the resolution being abnormal
ubuntu-drivers devicesYou can check the recommended driver version
Take my output as an example
driver : nvidia-driver-495 - third-party non-free
driver : nvidia-driver-535 - third-party non-free recommended
driver : nvidia-driver-510 - third-party non-free
driver : nvidia-driver-465 - third-party non-free
You can see that 535 is the adapted driver version for the current graphics card
Then open Software&Updates to install the corresponding version
After restarting the computer, if the display is normal, you should be done. But I once encountered a problem, that is, I had used the recommended version of the graphics card driver, but could not light up the secondary screen, and then downgraded the version to solve it. All in all, if there is a problem, it is most likely a problem with the graphics card driver. Try changing to a few more versions.
Enter in the terminalnvidia-smi. If no error is reported, it will be ok. In the output information, you can see the current driver version and the highest supported version of CUDA. This will be required when installing CUDA later. Note that the version cannot exceed this.
1.2 CUDA installation
https://developer.nvidia.com/cuda-toolkit-archive
Select the appropriate CUDA version from this link, take mine as an example
After selecting, you can see the installation instructions below and just execute them one by one. But there was a problem when I installed it. It is possible that the system will automatically upgrade you to the latest driver. Some libraries require a special CUDA version, so we can change the last line of the command to install the specified version of CUDA.sudo apt-get -y install cuda-xxx
1.3 Configure environment variables
Terminal inputnvcc -V to check whether CUDA is installed successfully. If there is a problem that the command cannot be found, don’t worry. We also need to configure the CUDA environment variables
Terminal execution sudo gedit ~/.bashrc, add (the path is your own) at the end of the file
export CUDA_HOME=/usr/local/cuda-11.8
if [[ ":$PATH:" != *":/usr/local/cuda-11.8/bin:"* ]]; then
export PATH=$PATH:/usr/local/cuda-11.8/bin
fi
if [[ ":$LD_LIBRARY_PATH:" != *":/usr/local/cuda-11.8/lib64:"* ]]; then
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda-11.8/lib64
fi
Entersource ~/.bashrc again to make the changes take effect. This time when executingnvcc -V, there is the following output, which means the installation is successful.
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:33:58_PDT_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0
2. 3D gaussian installation
The installation of this project is not difficult, and the problems are basically related to CUDA
This part is a summary of the official website bonushttps:/ /github.com/graphdeco-inria/gaussian-splatting
git clone [email protected]:graphdeco-inria/gaussian-splatting.git --recursive
The last one - recursive is more important. If you don't add it, some things won't be installed.- Enter the cloned project directory and execute the following command if anaconda has been installed.
conda env create --file environment.yml
conda activate gaussian_splatting
When installing two sub-modules, some errors may be reported. These errors are hidden in long texts and sometimes are not color-coded, so you need to carefully find the error points, such as some local libraries are missing and the CUDA environment variable is not set. To the same category.
Requires great attention! ! ! ! 3dgs has special requirements for the CUDA version. The full version of CUDA installed locally is 11.8, and the castrated version installed in the conda environment for use by pytorch is 11.6 (this version does not include nvcc, Some complex functions are not available), the official explanation is as follows:
Hi,
there’s two different things. One is the full CUDA SDK, including the compiler (NVCC). We need it to build the PyTorch extensions that we wrote ourselves and that the optimizer uses. The other is the CUDA runtime that PyTorch is built against (the latest CUDA runtime that PyTorch 1.12 works with is 11.6). If you install PyTorch with CUDA with Conda, it installs a small subset of the full CUDA SDK that cannot do compilation. If you install the full 11.6 SDK and try to compile our extensions with it, it can fail because of a known issue with C++14 support in the 11.6 CUDA compilers.
training codepython train.py -s <path to COLMAP or NeRF Synthetic dataset>
3. Viewer installation
According to the official statement, it is currently available on windows\ubuntu20.04 and 22.04. Other versions are unknown.
sudo apt install -y libglew-dev libassimp-dev libboost-all-dev libgtk-3-dev libopencv-dev libglfw3-dev libavdevice-dev libavcodec-dev libeigen3-dev libxxf86vm-dev libembree-dev
# Project setup
cd SIBR_viewers
git checkout fossa_compatibility #如果是22.04就不需要加这行命令
cmake -Bbuild . -DCMAKE_BUILD_TYPE=Release # add -G Ninja to build faster
cmake --build build -j24 --target install
Test with the trained model downloaded from the official website
./<SIBR install dir>/bin/SIBR_gaussianViewer_app -m <path to trained model>
