Introduction to BEVFusion, environment configuration and installation, and handling of various error reports encountered

Introduction to BEVFusion

Regarding the multi-modal fusion of point cloud projection to image and the multi-modal fusion of image projection to point cloud, the former will lose spatial geometric information, and the latter will lose image semantic information. These two point-level multi-modal fusion methods Neither of them can handle the target detection and map segmentation problems well at the same time.

BEVFusion is an efficient and versatile multi-task multi-sensor fusion framework. It unifies multi-modal features in a shared bird's-eye view (BEV) representation space and fuses image and point cloud features in the BEV space. This network is composed of a fully convolutional network and improves the efficiency of 2D-3D view conversion. Optimization and improvement have reduced latency by more than 40 times and greatly improved computing speed.

The network structure of BEVFusion is shown in the figure below:

BEVFusion environment configuration and installation

The blogger’s own hardware configuration is as follows:

• Linux (Ubuntu 20.04)
• NVIDIA GeForce RTX 3090
• NVIDIA graphics card driver version: 12.0
• CUDA version：11.3

Configure the python environment required by BEVFusion (if the CUDA version is different, some modifications need to be made during installation):

# 创建名为bevfusion的conda环境
conda create -n bevfusion python=3.8 -y
conda activate bevfusion
# 安装pytorch
conda install pytorch==1.10.0 torchvision==0.11.0 torchaudio==0.10.0 cudatoolkit=11.3 -c pytorch -c conda-forge
# 安装mmcv
pip install mmcv-full==1.4.0 -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.10.0/index.html
# 安装mmdetection
pip install mmdet==2.20.0
# 安装nuscenes数据集工具、软件开发包
pip install nuscenes-dev-kit
# 安装mpi4py
pip install mpi4py==3.0.3
# 安装torchpack
pip install torchpack
# 安装numba
pip install numba
cd bevfusion
# 编译、安装mmdetection3d
python setup.py develop

Error reporting and resolution

This section mainly introduces some errors and solutions encountered by bloggers during installation, training or testing.

Error one

During training or testing, spconv reports an error:

RuntimeError: /XXX/XXX/XXX/bevfusion/mmdet3d/ops/spconv/src/indice_cuda.cu 124
cuda execution failed with error 2

The solution is:
1. Run in the bevfusion folder python setup.py developto install the adapted spconv version.
2. If you have tried step (1) but still get an error, it may be caused by insufficient video memory. Try changing to a graphics card with larger video memory or reducing the batch_size.

Error 2:
I encountered an error during training:

TypeError: FormatCode() got an unexpected keyword argument 'verify'

The solution is to install an adapted version of yapf: pip install yapf==0.40.1.

Error three

AttributeError: module 'distutils' has no attribute 'version'
Primary job terminated normally, but 1 process returned a non-zero exit code. Per user-direction, the job has been aborted.

The solution is to install an adapted version of setuptools: pip install setuptools==59.5.0.

Error four

Traceback (most recent call last):
  File "/home/XXX/bevfusion/tools/train.py", line 91, in <module>
    main()
  File "/home/XXX/bevfusion/tools/train.py", line 24, in main
    dist.init()
  File "/home/XXX/anaconda3/envs/BEVFusionEnv/lib/python3.8/site-packages/torchpack/distributed/context.py", line 23, in init
    master_host = 'tcp://' + os.environ['MASTER_HOST']
  File "/home/XXX/anaconda3/envs/BEVFusionEnv/lib/python3.8/os.py", line 673, in __getitem__
    raise KeyError(key) from None
KeyError: 'MASTER_HOST'

The solution is to add environment variables: export MASTER_HOST=IP:port.

Error five

ModuleNotFoundError: No module named 'mmcv._ext'

The reason for the error was that mmcv=1.4.0 was installed incorrectly.
The solution is to install mmcv-full==1.4.0, remember not to omit full.

Error 6:
The pth file cannot be downloaded when executing the download_pretrained.sh file, and an error message appears: ERROR 404: Not Found.

The solution is to download it on Google Cloud Drive, the link is as follows: https://drive.google.com/drive/folders/1Jru7VTfgvFF949DlP1oDfriP3wrmOd3c .

Error seven

_configtest.c:2:10: fatal error: mpi.h: No such file or directory
       #include <mpi.h>
                ^~~~~~~
      compilation terminated.
      failure.
      removing: _configtest.c _configtest.o
      error: Cannot compile MPI programs. Check your configuration!!!
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for mpi4py
  Running setup.py clean for mpi4py
Failed to build mpi4py
ERROR: Could not build wheels for mpi4py, which is required to install pyproject.toml-based projects

The solution is to install libopenmpi-dev first, and then install mpi4py:

sudo apt install libopenmpi-dev
pip install mpi4py==3.0.3

Error eight

cannot import name 'ball_query_ext' from 'mmdet3d.ops.ball_query'

The solution is

cd bevfusion
python setup.py develop

Error nine

AttributeError: module 'numpy' has no attribute 'int'.
`np.int` was a deprecated alias for the builtin `int`. To avoid this error in existing code, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:
    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations

The solution is that the installed version of numpy is too high, install a lower version of numpy, for example:

pip install numpy==1.23.1

Report error ten

N > 0 assert faild. CUDA kernel launch blocks must be positive, but got N= 0

Solution:

# cuda是10.2版本
pip install spconv-cu102
# cuda是11.3版本
pip install spconv-cu113