This article shares how to use docker to obtain Nvidia images, including cuda10, cuda11 and other different versions, cudnn7, cudnn8, etc., to quickly build a deep learning environment.
1. Go to the docker hub official website and check which Nvidia images are available
https://hub.docker.com/r/nvidia/cuda/tags?page=2&name=11.3
Here you can enter the cuda version such as 11.6, or filter out related images:
https://hub.docker.com/r/nvidia/cuda/tags?page=1&name=11.6
Next to it is the sorting method of image names:
2. Pull the image to local
Select the desired image, for example: 11.3.1-cudnn8-devel-ubuntu20.04
Click the copy button on the right
Copy to command:
docker pull nvidia/cuda:11.3.1-cudnn8-devel-ubuntu20.04Then go to the command terminal and execute:
At this time, we started to pull the nvidia/cuda:11.3.1-cudnn8-devel-ubuntu20.04 image to the local area. You need to wait for a while.
3. View the image and open the image
After pulling the image, use the docker images command to view the image status:
docker images
You can see that the nvidia/cuda:11.3.1-cudnn8-devel-ubuntu20.04 image is local, and the image size is 8.95G.
Open the image (regular mode--supports the use of GPU)
docker run -i -t --gpus all nvidia/cuda:11.3.1-cudnn8-devel-ubuntu20.04 /bin/bash
Open the image (enhanced mode--supports using GPU, mapping directory, and setting memory)
docker run -i -t -v /home/liguopu/:/guopu:rw --gpus all --shm-size 16G nvidia/cuda:11.3.1-cudnn8-devel-ubuntu20.04 /bin/bashUsually, after entering the docker environment, the files created or generated will be "automatically destroyed" after exiting the docker environment; or if you want to run a program on the local host, you find that it cannot be found in the docker environment.
We can map a directory on the local host to the docker environment by mapping the directory, so that the generated files will be retained on the local host.
Use -v to map the local host directory /home/liguopu/ to the /guopu directory in the docker environment; its permissions are rw, which means it can read and write.
The default allocation of internal parameters is very small, which is not enough when training the model. You can set it through parameters: For example, my computer has 32G internal parameters and I want to put 16G into docker for use. Set it to --shm-size 16G.
4. Test images cuda and cudnn
Use nvidia-smi to check graphics card information
The CUDA Version you see here: 11.4 is consistent with the host, but what we want is 11.3, so don’t worry, keep reading.
Use the nvcc -V command to view the actual cuda version. Building a deep learning environment later also relies on cuda 11.3 here.
Use the ls /usr/local/ command to check the cuda installation path
Let’s start building a deep learning environment
5. Install conda environment
The default system image may not have conda or python. We can install Anaconda to build a deep learning environment.
There are two download addresses to choose from, namely the official download address and the Tsinghua University open source software mirror station.
Official download address: Free Download | Anaconda
If you feel that downloading from the official address is slow, you can try the download address of Tsinghua University:
Tsinghua University Open Source Software Mirror Station: Index of /anaconda/archive/ | Tsinghua University Open Source Software Mirror Station | Tsinghua Open Source Mirror
For example, choose: Anaconda3-2023.07-2-Linux-x86_64.sh
Install Anaconda3
Go to the location of the downloaded file, open a terminal in the file directory, and use bash to perform the installation:
bash Anaconda3-2023.07-2-Linux-x86_64.sh
Installation process:
1) Enter the Enter key to confirm installation;
2) Read the Anaconda End User License Agreement and press Enter to finish browsing the information;
3) Ask us if we accept the agreement, we can only accept it, enter yes; (Do you accept the license terms? [yes|no])
4) When prompted to install into the following location, press Enter to confirm; (Anaconda3 will now be installed into this location:)
5) Whether to add environment variables, usually choose yes; this choice depends on your own situation. If you often use the conda environment to develop, it is recommended to choose yes (Do you wish the installer to initialize Anaconda3 in your /home/linuxidc/.bashrc? [yes |no])
Start installation page
Installation process:
Complete the conda installation:
Use another terminal to open the container created by this image:
If you can see the initial environment for conda, it means that conda is installed successfully.