GPU computing

Part of the study notes of "Practice Deep Learning pytorch" is only for your own review.

GPU computing

For complex neural networks and large-scale data, the use of CPU to calculate may be 不 efficient. In this section, we will introduce how to use a single NVIDIA GPU to calculate. So you need to make sure that you have installed the PyTorch GPU version. After the preparations are completed, you can use the nvidia-smi command to view the graphics card information.

!nvidia-smi # 对Linux/macOS⽤用户有效 

Output:

Sun Mar 17 14:59:57 2019
+------------------------------------------------------------------
-----------+
| NVIDIA-SMI 390.48 Driver Version: 390.48
|
|-------------------------------+----------------------+-----------
-----------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile
Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util
Compute M. |
|===============================+======================+===========
===========|
| 0 GeForce GTX 1050 Off | 00000000:01:00.0 Off |
N/A |
| 20% 36C P5 N/A / 75W | 1223MiB / 2000MiB | 0%
Default |
+-------------------------------+----------------------+-----------
-----------+
+------------------------------------------------------------------
-----------+
| Processes:
GPU Memory |
| GPU PID Type Process name
Usage ||==================================================================
===========|
| 0 1235 G /usr/lib/xorg/Xorg
434MiB |
| 0 2095 G compiz
163MiB |
| 0 2660 G /opt/teamviewer/tv_bin/TeamViewer
5MiB |
| 0 4166 G /proc/self/exe
416MiB |
| 0 13274 C /home/tss/anaconda3/bin/python
191MiB |
+------------------------------------------------------------------
-----------+

Computing equipment

PyTorch can specify storage and computing devices , such as a CPU that uses memory or a GPU that uses video memory . By default, PyTorch will create data in memory, and then use the CPU to calculate.
Use torch.cuda.is_available() to see if the GPU is available:

import torch
from torch import nn
torch.cuda.is_available() # 输出 True

View the number of GPUs量:

torch.cuda.device_count() # 输出 1

View the current GPU index number, the index number starts from 0:

torch.cuda.current_device() # 输出 0 

View the GPU name according to the index number:

torch.cuda.get_device_name(0) # 输出 'GeForce GTX 1050' 

GPU computing by TENSOR

By default, Tensor will be stored in memory. Therefore, every time we printed Tensor before, we couldn't see the GPU related logo.

x = torch.tensor([1, 2, 3])
x

Output: tensor([1, 2, 3]) 

Use .cuda () to convert (copy) the Tensor on the CPU to the GPU . If there are multiple GPUs, we use .cuda(i) to represent the i-th GPU and the corresponding video memory (starting from 0) and cuda(0) and cuda() are equivalent.

x = x.cuda(0)
x

Output:

tensor([1, 2, 3], device='cuda:0') 

We can view the device where the Tensor is located through the device attribute of the Tensor.

x.device 

Output: device(type='cuda', index=0) 

We can directly specify the device when creating it.

device = torch.device('cuda' if torch.cuda.is_available() else'cpu')
x = torch.tensor([1, 2, 3], device=device)
# or
x = torch.tensor([1, 2, 3]).to(device)
x

If you perform operations on the data on the GPU, the result is still stored on the GPU.

y = x**2
y

Output: tensor([1, 4, 9], device='cuda:0') 

It should be noted that the data stored in the same location cannot be directly entered into the calculation. That is, the data stored on the CPU can be directly calculated with the data stored on the GPU, and the data located on the same GPU can also be directly calculated.

z = y + x.cpu() 

Will report an error:

RuntimeError: Expected object of type torch.cuda.LongTensor but
found type torch.LongTensor for argument #3 'other'

Model GPU calculation

Similar to Tensor, PyTorch models can also be converted to GPU via .cuda . We can view the device storing the model by checking the device attribute of the model's parameters.

net = nn.Linear(3, 1)
list(net.parameters())[0].device

Output:

device(type='cpu') 

The viewable model is on the CPU, convert it to the GPU:

net.cuda()
list(net.parameters())[0].device

Output: device(type='cuda', index=0) 

Similarly, we need to ensure that the Tensor and model input by the model are on the same device, otherwise an error will be reported.

x = torch.rand(2,3).cuda()
net(x)

Output:

tensor([[-0.5800],
[-0.2995]], device='cuda:0', grad_fn=<ThAddmmBackward>)

summary

• PyTorch can specify storage and computing devices, such as a CPU that uses memory or a GPU that uses video memory. By default, PyTorch will create data in memory, and then use the CPU to calculate.
• PyTorch requires all input data for calculation to be in the internal memory or the video memory of the same graphics card