Ubuntu configures deep learning environment (TensorFlow and pyTorch)

---

Preface: It is best to go here to see the CUDA version corresponding to TensorFlow and install it according to the required version. I did not install it according to this requirement because there are other programs that require 11.6.

1. CUDA installation

1.1 Install graphics card driver

View recommended graphics cards

ubuntu-drivers devices

Install the recommended graphics card version. The recommended version here is version 535.

sudo apt-get install nvidia-driver-535

You need to select a version or the kernel will be updated automatically. But when I install 535, the screen goes black, so I can only install 510. Since it is a newly installed system, updating the kernel will not have much impact. In addition, it will not work if the installed version is too low, because the minimum version is required to install CUDA below.

1.2 CUDA installation

Use the manual installation method:

nvidia-smi

Check that the highest CUDA version supported by the graphics card driver is 12.1. Go to cuda-toolkit-archive , select the required CUDA version to download, as shown below, select runfile (local), and use the generated instructions to download and install.

Press Enter to cancel the installation of the graphics card driver, and then select the final installation:

After successful installation, configure the environment and add environment variables at the end of the .bashrc file:

sudo gedit ~/.bashrc
# 添加以下内容：
export PATH=/usr/local/cuda-11.6/bin${
    
    PATH:+:${
    
    PATH}} 
export LD_LIBRARY_PATH=/usr/local/cuda-11.6/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

Or enter the following command in the terminal to add:

# Taken from: https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#post-installation-actions
echo 'export PATH=/usr/local/cuda-11.6/bin${PATH:+:${PATH}}' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda-11.6/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc

Verify successful installation

nvcc -V

1.3 Install cuDNN

After successfully installing CUDA, download the cuDNN corresponding version of CUDA from the official website and go to cudnn-archive (you need to register an account). Here I chose the 8.9.4 version of cuDNN (Local Installer for Linux x86_64 (Tar)), which corresponds to 11.x version of CUDA:

After the download is complete, unzip it in the directory where the file is located, and pay attention to the name of the file you downloaded.

tar -xvf cudnn-linux-x86_64-8.9.4.25_cuda11-archive.tar.xz

After decompression is completed, copy and transfer some files to /usr/local (note: the name of the downloaded file yourself, cudnn8.0 and above will update the version information to the cudnn_version.h file, you also need to copy this file, otherwise when verifying There will be no response):

cd cudnn-linux-x86_64-8.9.4.25_cuda11-archive
sudo cp include/cudnn.h /usr/local/cuda/include/ 
sudo cp lib/libcudnn* /usr/local/cuda/lib64/ 
sudo chmod a+r /usr/local/cuda/include/cudnn.h
sudo chmod a+r /usr/local/cuda/lib64/libcudnn*
sudo cp include/cudnn_version.h /usr/local/cuda/include/
cat /usr/local/cuda/include/cudnn_version.h | grep CUDNN_MAJOR -A 2  #验证

2. Anaconda installation

Enter the Anaconda official website and click Download (Anaconda will download the corresponding version according to the system used to access the web page. For example, I downloaded Anaconda3-2023.03-Linux-x86_64.sh here)

Install Anaconda

bash Anaconda3-2023.03-Linux-x86_64.sh

(1) View the installation agreement, press Enter until it appears Do you accept the license terms? [yes|no], enter yesto continue the installation;
(2) After entering yes, you will be prompted to confirm the installation location, click here Enter, the default is enough;
(3) Initialize Anaconda, this step only needs to follow the prompts Just enter yes;

Restart the terminal to enter the conda basic environment. You can check the python location and version in this environment:

If you want the conda basic environment not to be activated when starting the terminal, set auto_activate_basethe parameter to false:

conda config --set auto_activate_base false

If you want to enter conda's base environment later, you only need to activate it using the conda command:

conda activate base

Commonly used conda commands:

• Create conda environment

conda create --name 环境名 包名（多个包名用空格分隔）
# 例如：conda create --name my_env python=3.7 numpy pandas scipy

• Activate (switch) the conda environment

conda activate 环境名
# 例如：conda activate bas

• Display the created conda environment

conda info --envs
# 或者：conda info -e，亦或者conda env list

• Delete the specified conda environment,

# 通过环境名删除
conda remove --name 要删除的环境名 --all

# 通过指定环境文件位置删除（这个方法可以删除不同位置的同名环境）
conda remove -p 要删除的环境所在位置 --all
# 例如：conda remove -p /home/zard/anaconda3/envs/MaskRCNN --all

3. Install TensorFlow and pyTorch

3.1 Install pyTorch

Enter the pyTorch official website ( https://pytorch.org/ ), scroll down, and select your environment to generate the installation command:

Copy the installation command (I changed it to 11.6):

conda install pytorch torchvision torchaudio pytorch-cuda=11.6 -c pytorch -c nvidia

Enter y to install (you may need to access the Internet scientifically):

Test whether it can be used:

ipython

import torch
torch.cuda.is_availbale()

3.2 Install TensorFlow2

# 创建一个虚拟环境
conda create -n tensorflow-gpu python=3.7
# 激活环境
conda activate tensorflow-gpu

pip install ipython
pip install tensorflow -U

Be careful not to install tensorflow-gpu. The following error will be reported. According to the prompt message, you can also use the GPU when installing tensorflow.
Both methods allow your TensorFlow installation to use GPU acceleration ( in fact , as of TensorFlow 2.1, both packages are functionally identical), specifically:

• pip install --upgrade tensorflowWhen you install TensorFlow using This is because TensorFlow is a general-purpose framework that can run on both CPU and GPU, but requires the correct configuration and dependencies to take advantage of the GPU.
• When you install the TensorFlow-GPU version using pip install --upgrade tensorflow-gpu, it is already specifically configured to take advantage of GPU acceleration and requires no additional configuration.

Regardless of which method you choose, as long as the requirements for the GPU driver, CUDA toolkit, and cuDNN library are met, TensorFlow should be able to properly use the GPU for acceleration.

You can verify that GPU acceleration is enabled by checking the number of available GPUs:

import tensorflow as tf
tf.test.is_gpu_available()
print("Num GPUs Available: ", len(tf.config.experimental.list_physical_devices('GPU')))

4. Install pyCharm

4.1 Installation of pyCharm

Enter the official website ( https://www.jetbrains.com.cn/en-us/pycharm/ ) to download pycharm

, click download, scroll down, and download the Community version.

After the download is complete, unzip it, enter the bin directory, and run the .sh file

cd pycharm-community-2023.2.1/bin/
sh pycharm.sh

After the interface starts, click the settings icon in the lower left corner of the opened Pycharm software interface and select Create Deasktop Entry. Close Pycharm, click on the lower left corner to display the application, find Pycharm, and start using it.
Click to enter plugins, select Marketplace, search for chinese, find the Chinese language pack (there is a "汉" character on the icon) to install it, and restart pycharm after installation. Can

4.2 Python interpreter associated with anaconda

After creating a new project, open settings and select python interpreter:

add python interpreter, select Conda environment, select the virtual environment we created before:

you can see that the libraries we installed are already there