Introduction to spconv, environment configuration and installation, and handling of various error reports encountered

This blog will introduce spconv, how to install spconv 1.x and spconv2.x, and handle various error reports encountered. For more details, please refer to the official GitHub link.

Introduction to spconv

spconv is a spatial sparse convolution library, a project that provides highly optimized sparse convolution implementation and tensor core support. spconv is designed to efficiently handle sparse data containing a large number of zero elements, and is mainly used for convolution operations on 3D point clouds.

spconv installation

The blogger’s own hardware configuration is as follows:

• Linux (Ubuntu 20.04)
• NVIDIA GeForce RTX 3090
• NVIDIA graphics card driver version: 12.0
• CUDA version：11.3

First install pytorch, cuda and other related software packages:

conda create --name env_name python=3.7 cmake=3.22.1
conda install pytorch==1.10.1 torchvision==0.11.2 torchaudio==0.10.1 cudatoolkit=11.3 -c pytorch -c conda-forge
conda install cudnn -c conda-forge
conda install boost

Open the bashrc file: vim ~/.bashrc, set environment variables:

export PATH=/usr/local/cuda/bin:$PATH
export CUDA_HOME=/usr/local/cuda/bin:$CUDA_HOME
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
export C_INCLUDE_PATH=$C_INCLUDE_PATH:/[$YourUserName]/anaconda3/envs/[$YourEnvName]/include
export CPLUS_INCLUDE_PATH=$CPLUS_INCLUDE_PATH:/[$YourUserName]/anaconda3/envs/[$YourEnvName]/include
export LIBRARY_PATH=$LIBRARY_PATH:/[$YourUserName]/anaconda3/envs/[$YourEnvName]/lib
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/[$YourUserName]/anaconda3/envs/[$YourEnvName]/lib

Install spconv 1.x from source code

spconv 1.x has been officially abandoned and will not be maintained. It is recommended to use spconv 2.x. Moreover, spconv 2.x has been optimized compared to spconv 1.x, and its speed and efficiency have been improved. If you have to install spconv 1.x due to version requirements, you need to compile and install the source code.
The following are the official steps to install spconv 1.2.1:

# STEP 1: get source code
git clone https://github.com/traveller59/spconv.git 
cd spconv
git checkout v1.2.1
git submodule update --init --recursive 

# STEP 2: compile
python setup.py bdist_wheel

# STEP 3: install
cd ./dist
pip install spconv-1.2.1-cp37-cp37m-linux_x86_64.whl

# check if is successfully installed
python 
import spconv

pip install spconv 2.x

spconv 2.x currently supports installation through pip install. You can choose to run the following commands according to your own environment:

# 预先安装cumm和timm
pip install cumm timm

# cpu
pip install spconv
# cuda 10.2
pip install spconv-cu102
# cuda 11.3
pip install spconv-cu113
# cuda 11.4
pip install spconv-cu114
# cuda 11.6
pip install spconv-cu116
# cuda 11.7
pip install spconv-cu117
# cuda 11.8
pip install spconv-cu118
# cuda 12.0
pip install spconv-cu120

Error handling

Here we mainly list the errors and solutions when installing spconv 1.x.

Error one

error This file requires compiler and library support for the ISO C++ 2011 standard. This support is currently experimental, and must be enabled with the -std=c++11 or -std=gnu++11 compiler options.

Enter gcc --versionthe gcc version 4.8.5. The solution is to install the 5.4.0 version of gcc. Here, I am using the conda virtual environment installation:

conda install https://anaconda.org/brown-data-science/gcc/5.4.0/download/linux-64/gcc-5.4.0-0.tar.bz2

A pitfall of this package is that the soft links miniconda3/envs/env_name/libunder the file point to the dynamic link library, and this soft link will overwrite the original soft link in the environment. In other words, the dynamic links provided by this package will be used in the future. Library.libstdc++.solibstdc++.so.6libstdc++.so.6.0.21

This will cause if your pytorch version is higher, an libstdc++.so.6.0.21error will be reported because the version is older:

ImportError: /xxx/miniconda3/envs/env_name/lib/libstdc++.so.6: version `GLIBCXX_3.4.22' not found (required by /xxx/miniconda3/envs/env_name/lib/python3.7/site-packages/open3d/open3d.cpython-37m-x86_64-linux-gnu.so)

The solution is to enter miniconda3/envs/env_name/liband then point the soft link to a higher version of the dynamic link library, for example libstdc++.so, libstdc++.so.6in the same directory libstdc++.so.6.0.32. The command is as follows:

# 先删除原来的软连接
rm libstdc++.so
rm libstdc++.so.6
# 建立新的软连接
ln -s libstdc++.so.6.0.32 libstdc++.so
ln -s libstdc++.so.6.0.32 libstdc++.so.6

Reference link: https://blog.csdn.net/j___t/article/details/107308883

Error two

/xxx/spconv/include/tensorview/tensorview.h:741:23: error: template declaration of ‘constexpr const char* const tv::type_s’
 constexpr const char *type_s = detail::TypeToString<T>::value;
                       ^
/xxx/spconv/include/tensorview/tensorview.h: In member function ‘std::string tv::TensorView<T, Rank, PtrTraits, Tindex>::repr(Os&) const’:
/xxx/spconv/include/tensorview/tensorview.h:1124:26: error: ‘type_s’ was not declared in this scope
       ss << "Tensor[" << type_s<T> << "]" << std::endl;
                          ^
/xxx/spconv/include/tensorview/tensorview.h:1124:34: error: expected primary-expression before ‘>’ token
       ss << "Tensor[" << type_s<T> << "]" << std::endl;
                                  ^
/xxx/spconv/include/tensorview/tensorview.h:1124:36: error: expected primary-expression before ‘<<’ token
       ss << "Tensor[" << type_s<T> << "]" << std::endl;
                                    ^
/xxx/spconv/include/tensorview/tensorview.h:1135:24: error: ‘type_s’ was not declared in this scope
     ss << "Tensor[" << type_s<T> << "]: shape=" << shape()
                        ^
/xxx/spconv/include/tensorview/tensorview.h:1135:32: error: expected primary-expression before ‘>’ token
     ss << "Tensor[" << type_s<T> << "]: shape=" << shape()
                                ^
/xxx/spconv/include/tensorview/tensorview.h:1135:34: error: expected primary-expression before ‘<<’ token
     ss << "Tensor[" << type_s<T> << "]: shape=" << shape()
                                  ^
/xxx/spconv/include/tensorview/tensorview.h: At global scope:
/xxx/spconv/include/tensorview/tensorview.h:1270:23: error: template declaration of ‘constexpr const char* const tv::detail::type_printf_format_v’
 constexpr const char *type_printf_format_v = TypePrintfFormat<T>::value;
                       ^
/xxx/spconv/include/tensorview/tensorview.h: In function ‘void tv::printTensorView(tv::TensorView<T, Rank, PtrTraits, Tindex>)’:
/xxx/spconv/include/tensorview/tensorview.h:1343:34: error: ‘type_printf_format_v’ is not a member of ‘tv::detail’
   return printTensorView(tensor, detail::type_printf_format_v<Traw>);
                                  ^
/xxx/spconv/include/tensorview/tensorview.h:1343:67: error: expected primary-expression before ‘>’ token
   return printTensorView(tensor, detail::type_printf_format_v<Traw>);
                                                                   ^
/xxx/spconv/include/tensorview/tensorview.h:1343:68: error: expected primary-expression before ‘)’ token
   return printTensorView(tensor, detail::type_printf_format_v<Traw>);
                                                                    ^
/xxx/spconv/include/tensorview/tensorview.h: In function ‘void tv::printTensorView(const T*, tv::Shape)’:
/xxx/spconv/include/tensorview/tensorview.h:1349:26: error: ‘type_printf_format_v’ is not a member of ‘tv::detail’
                          detail::type_printf_format_v<Traw>);
                          ^
/xxx/spconv/include/tensorview/tensorview.h:1349:59: error: expected primary-expression before ‘>’ token
                          detail::type_printf_format_v<Traw>);
                                                           ^
/xxx/spconv/include/tensorview/tensorview.h:1349:60: error: expected primary-expression before ‘)’ token
                          detail::type_printf_format_v<Traw>);

The solution is to delete the build folder directly and re-run it python setup.py bdist_wheel. If it is not solved, you can try copying boost/. to spconv/include/, but I have not tried this method and I am not sure whether it can be solved.

Error three

The following two errors are caused by the CUDA environment variables not being found or incorrect:

CMake Error at /disk1/zhuhe/miniconda3/envs/env_name/lib/python3.7/site-packages/cmake/data/share/cmake-3.28/Modules/CMakeDetermineCUDACompiler.cmake:270 (message):
  Failed to detect a default CUDA architecture.



  Compiler output:

Call Stack (most recent call first):
  CMakeLists.txt:9 (project)

CMake Error at /xxx/miniconda3/envs/env_name/lib/python3.7/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:87 (message):
  FindCUDA says CUDA version is (usually determined by nvcc), but the CUDA
  headers say the version is ERROR: ld.so: object

  10.2.  This often occurs when you set both CUDA_HOME and
  CUDA_NVCC_EXECUTABLE to non-standard locations, without also setting PATH
  to point to the correct nvcc.  Perhaps, try re-running this command again
  with PATH=/usr/local/cuda-10.2/bin:$PATH.  See above log messages for more
  diagnostics, and see https://github.com/pytorch/pytorch/issues/8092 for
  more details.
Call Stack (most recent call first):
  /xxx/miniconda3/envs/env_name/lib/python3.7/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:88 (include)
  /xxx/miniconda3/envs/env_name/lib/python3.7/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
  CMakeLists.txt:22 (find_package)

Open the bashrc file: vim ~/.bashrc, and set the CUDA-related environment variables.

export PATH=/usr/local/cuda/bin:$PATH
export CUDA_HOME=/usr/local/cuda
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH