Article Directory
1. Problem description
When running the program, the runtime cudnn version problem is encountered, and the program reports an error, as shown in the figure below.
ie : Loaded runtime cudnn library: 7103 (version 7100 compatible), but the source code was compiled with 7005 (version 7000 compatible). If using a binary install, upgrade your cudnn library to match. If building from source, make sure that the library loaded at runtime matches the coroutine version specified during compile configuration.
That is to say, the version of cudnn in the environment where the code runs is 7.1.3, but the version of cudnn required by the source code is 7.0.5, and the version of cudnn in the program environment must be adjusted.
Two, the solution
1. Process description
Refer to the following information to get two solutions. One is to upgrade the TF version to 1.8.0 (the configuration environment is TF1.4.1), but the TF on the code README reproduced by the author is 1.4.1. I don’t want to run the program later. There are more problems.
1. Running Loaded runtime CuDNN library: 7103 (compatibility version 7100) but source was compiled with 7003 2. Building a
deep learning environment (2): Different versions of Ubuntu gcc, CUDA, and cuDNN coexist, switching solutions
3. Install and switch multiple versions of cuda on ubuntu
4. Pytorch-cuDNN version mismatch: PyT orch was compiled against 7005 but linked against 7103
So far, a new problem has appeared. The program error shows that the running cudnn7.1.3 and the environment configuration cudnn6.0 are inconsistent. Reference 4 knows that the virtual environment is established by anaconda, and there is cudnn7.1.3 in the lib folder, so the running program does not use the configured cudnn6.0. (That is, cudnn7.1.3 exists in /home/ubuntu/anaconda3/envs/py27_tf141/lib)
The following command is used to query the cudnn version, but this query is the locally installed cudnn version. It turns out that the virtual environment created by anaconda does not necessarily use the locally installed cudnn.
cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2
(You can directly see the solution steps below) A reminder to myself:
There are multiple versions of cuda on the author’s computer. According to reference 3, the cuda environment variable used has been set, and the cuda link is pointed to cuda8.0, that is to say, the cuda in the environment variable path file is the installed cuda8.0 folder.
2. Solution steps
① Download cuDNN7.0.5, and put the cuda folder obtained after decompressing the compressed package at home for later use.
The process refers to installing CUDA and cuDNN on Ubuntu 16.04 .
② Add the cuDNN7.0.5 file to the virtual environment created by anaconda. The author named the virtual environment py27_tf141.
Right-click on home to open the terminal, and enter the following command. The destination path is the path of the include and lib folders of the virtual environment, which must meet your actual situation.
sudo cp cuda/include/cudnn.h /home/ubuntu/anaconda3/envs/py27_tf141/include
sudo cp cuda/lib64/libcudnn* /home/ubuntu/anaconda3/envs/py27_tf141/lib
sudo chmod a+r /home/ubuntu/anaconda3/envs/py27_tf141/include/cudnn.h /home/ubuntu/anaconda3/envs/py27_tf141/lib/libcudnn*
③ The soft link points to
the lib folder of the virtual environment, that is, /home/ubuntu/anaconda3/envs/py27_tf141/lib, and uses the following command to view cudnn related files and soft link points.
ls -al |grep cudnn
It can be seen that there are cudnn7.0.5 and cudnn7.1.3, libcudnn.so points to libcudnn.so.7, and libcudnn.so.7 points to libcudnn.so.7.0.5, which is the actual soft link.
Change the soft link to point to cudnn7.0.5 with the following command.
sudo ln -sf libcudnn.so.7.0.5 libcudnn.so
sudo ln -sf libcudnn.so.7.0.5 libcudnn.so.7
The above has solved the author's problem.
related records
1. Query cudnn version:
cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2
In the figure, the cudnn runtime version defined by the cudnn version is 7005, and the three-digit number corresponds to the version cuDNN7.0.5.
2. View the created soft connection command:
ll /usr/local/cuda/lib64/
At the same time, you can see what versions of cuda and cudnn exist.