situation
When testing the LLM Webui demo , after installing the dependency pip install bitsandbytes==0.40.0, python -m module name reported an error: CUDA Setup failed despite GPU being available. Please run the following command to get more information.
Solution
Download reference from git hub official repo
Requires manual installation:
Use the following steps to install manually:
pip uninstall bitsandbytes --yes # if you already installed it with pip
git clone https://github.com/TimDettmers/bitsandbytes #Please download manually if the network is not good
cd bitsandbytes
CUDA_VERSION=114 make cuda11x GPP=$(which g++) -j 2 #Change here to your cuda version, check it with nvidia-smi
pip install .
python check_bnb_install.py
python -c ‘import bitsandbytes’