Windows installs GPU environment CUDA, deep learning framework Tensorflow and Pytorch
1. CUDA is not installed and tensorflow is used to report an error
import tensorflow as tf
2022-03-06 15:14:38.869955: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
2022-03-06 15:14:38.870236: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2. Introduction to CUDA
First, you need to install the GPU environment, including cuda and cudnn.
Deep learning is essentially training deep convolutional neural networks.
cuda: The graphics card can complete parallel computing tasks, and all operations are relatively low-level and complex.
cudnn: On top of cuda, there is an SDK library dedicated to deep neural networks to accelerate the completion of specific deep learning operations. It is a GPU accelerated library for deep neural networks. It emphasizes performance, ease of use, and low memory overhead. NVIDIA cuDNN can be integrated into higher-level machine learning frameworks, such as caffe, tensorflow, pytorch, mxnet, etc. cudnn's simple plug-in design allows developers to focus on designing and implementing neural network models rather than tuning performance, while also enabling high-performance modern parallel computing on GPUs.
cuda is a series of low-level GPU operation libraries used to define parallel operations of graphics cards, and cudnn is an advanced GPU operation library customized for deep learning based on cuda.
Here our matching version is CUDA 11.0
3. Install CUDA
According to Nvidia, CUDA cores now offer concurrent execution of floating-point and integer operations to improve performance for compute-intensive workloads in modern games.
Query the matching relationship between Tensorflow version and CUDA
tensorflow_gpu-2.4.0
3.1 Download CUDA
https://developer.nvidia.com/cuda-downloads
Select Windows, open cmd to view the windows version
After installation, in Anaconda, enter nvcc -V to test
Anaconda's usage tutorial can be viewed in the previous article:
Python How to use and configure Anaconda Getting Started
nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Thu_Feb_10_19:03:51_Pacific_Standard_Time_2022
Cuda compilation tools, release 11.6, V11.6.112
Build cuda_11.6.r11.6/compiler.30978841_0
CUDA download link: https://developer.nvidia.com/cuda-toolkit-archive
cudnn download link: https://developer.nvidia.com/cuda-downloads
3.2 Install Tensorflow under CUDA
Activate the Anaconda virtual environment
conda activate tfenv_py37
conda install tensorflow-gpu
Python 3.7.4 (default, Aug 9 2019, 18:34:13) [MSC v.1915 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
import tensorflow as tf
2022-03-06 16:21:03.223773: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library cudart64_110.dll
Cannot dlopen some GPU libraries.Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
The latest version is installed, but the version does not match. You need to select the corresponding CUDA according to the CUDA version of the graphics card.
3.3 Testing Tensorflow
import tensorflow as tf
a = tf.constant(2)
Could not load dynamic library 'cudnn64_8.dll'; dlerror: cudnn64_8.dll not found
Download address: https://developer.nvidia.com/rdp/cudnn-download
3.4 Install CUDNN
Download link: https://developer.nvidia.com/rdp/cudnn-archive
Unzip
and copy to C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.0
3.5 Test CUDA with Tensorflow again
import tensorflow as tf
tf.test.gpu_device_name() # 显示显卡型号
print(tf.test.is_gpu_available()) # 提示True
Not creating XLA devices, tf_xla_enable_xla_devices not set
solution
os.environ['TF_XLA_FLAGS'] = '--tf_xla_enable_xla_devices' os.environ['TF_CPP_MIN_LOG_LEVEL']='2'
In fact, this is due to the new features of Tensorflow 2.4 version, which can be **directly ignored.** Look at the 2.4 version release and it will be clear at a glance. It is not a version correspondence problem mentioned by many bloggers. Going back to the old version will solve the symptoms but not the root cause.
If you need to use XLA, add TF_XLA_FLAGS=–tf_xla_enable_xla_devices to solve the warning.
4. Install pytorch
PyTorch's speed performance outperforms frameworks such as TensorFlow and Keras. PyTorch has the most elegant object-oriented design of all the frameworks.
PyTorch is mainly used for deep learning algorithm modeling and reasoning. In order to speed up algorithm training, it is generally necessary to use a computer with a GPU for Pytoch installation. In order for PyToch to use GPU resources, a GPU environment, including CUDA and CUDNN, needs to be installed.
Install Pytorch
conda install pytorch torchvision torchaudio cudatoolkit=11.0 -c pytorch
# CUDA 11.0
conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 cudatoolkit=11.0
pip install torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html