PyTorch installation + basics

PyTorch installation + basics

Datawhale202210——PyTorch （0、1、2）

Anaconda+CUDA+PyTorch+Jupyter notebook full installation + PyTorch introductory knowledge

---

---

foreword

Understanding and using PyTorch is an entry requirement and housekeeping skill for relevant practitioners and learners. In order to better learn and use PyTorch, I choose to go forward side by side with Datawhale's friends, so let's start this wonderful journey!

---

1. What is PyTorch?

Official definition:
1. PyTorch is a tensor library (tensor library) optimized for deep learning and using GPU and CPU. 2. PyTorch is a deep learning framework
developed by Facebook and based on Torch, which is developed from the uncommon Lua language to Python language .
Therefore, learners who are accustomed to using Python and want to explore deep learning are very fond of PyTorch, and have sent it to the position of the most popular deep learning framework. Currently, its Github also has 59.5k+ followers.

2. Installation of PyTorch (GPU version)

1. Anaconda installation

Official website portal: https://www.anaconda.com/
1) Select the appropriate version according to the python version and system version and download it directly.
2) Open the .exe file and follow the prompts to install, generally there is no pitfall.
3) Verification of successful installation: Enter the following command in cmd

save --version
python

2. CUDA installation

Official website portal: https://developer.nvidia.com/cuda-toolkit
(The official website will give priority to recommending downloading the latest version, but it is possible that PyTorch has not been updated to the latest version of CUDA < The predicted version does not correspond to possible problems, but I have not tried it yet >, if you want to download the historical version, you can directly search for the CUDA+ version number)
1) Select according to your needs and install it directly.
(Larger local files do not require further installation, and fast network downloads require continuous networking to complete subsequent downloads)
2) Install according to the prompts, and adjust the installation location appropriately.
(Installation details related to environment variables and other configurations will not be described in detail, please refer to the reference document for details)
3) To verify whether the installation is successful, enter the following command in cmd.

nvcc --version
set cuda

3.Pytorch installation

Official website portal: https://pytorch.org/
1) Specify the system version, CUDA version and other information, and select as needed.
(About using conda or pip, the author recommends using pip. Firstly, there are fewer pitfalls, and secondly, it may be more conducive to subsequent use. Conda is very prone to problems when installed, which leads to the problem that half of the downloads are stuck in the end. Change pip decisively, but The difference between the two has not yet been understood and experienced, and I hope that I can post a special explanation in the future)
2) Open Anaconda Prompt (anaconda3), and enter the command given on the official website. (The example is as follows, it is recommended to go to the official website to copy by yourself)

pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cu116

3) To speed up the download, you need to switch the mirror source. For specific tutorials, please refer to the reference document.
4) To verify whether the installation is successful, enter the following command:

python
import torch

4. jupyter notebook calls PyTorch

1) jupyter notebook is an application in Anaconda, which can be opened by Anaconda Prompt (anaconda3), just enter the following command:

jupyter notebook

But after opening it, I found that the hard-working PyTorch cannot be called by jupyter notebook, and the following steps need to be completed:
2) Enter the following command to install the plug-in: (note that the environment you need must be activated first)

conda install nb_conda
conda install ipykernel

3) Restart the jupyter notebook and you will find that there are more options for PyTorch, and the verification shows that it can be called normally.

3. Basic knowledge of PyTorch

1. Tensor
1) Basic definition and characteristics

• Tensor can be high-dimensional

• The basic unit of PyTorch operation

• Basic data definition and operation

• Support GPU computing, automatic derivation and other operations in PyTorch

2) PyTorch's tensor

data->mathematical representation
dtype->data type
Class coercion type, but not recommended

3) Related operations of tensor

Conversion between tensor and numpy, array

import numpy as np
g=np.array([[1,2,3],[4,5,6]])
h=torch.tensor(g)
print(h)
i=torch.from_numpy(g)
print(i)
j=h.numpy()
print(j)

View tensor dimension information

print（k.shape)
print(k.size())

An artifact that changes the shape of tensor

print（o.view((3,2)))
print(o.view(-1,2))

Tensor's broadcast mechanism

p=torch.arange(1,3).view(1,2)
print(p)
q=torch.arange(1,4).view(3,1)
print(q)
print(p+q)

Compress the dimension of tensor: squeeze

print(o)
r=o.unsqueeze(1)
print(r)
print(r.shape)

2. Automatic derivation
1) Introduction to Autograd

• torch.Tensor is the core class of Autograd. If you set its attribute .requires_grad to True, then it will track all operations on the tensor.
• In y.backward(), if y is a scalar, you don't need to pass in any parameters for backward(); otherwise, you need to pass in a Tensor with the same shape as y.
• Tensor and Function are interconnected to generate an acyclic graph (acyclic graph), which encodes the complete computation history.
• Each tensor has a .grad_fn attribute, which refers to the Function that created the Tensor itself (unless the tensor is manually created by the user, that is, the grad_fn of this tensor is None).

2) gradient

start backpropagation

out.backward()
print(x.grad)
tensor([[3., 3.],
        [3., 3.]])

second backpropagation

#grad在反向传播过程中是累加的每一次运行反向传播，梯度都会累加之前的梯度，所以一般在反向传播之前需把梯度清零。
out2 = x.sum()
out2.backward()
print(x.grad)

out3 = x.sum()
x.grad.data.zero_()
out3.backward()
print(x.grad)

tensor([[4., 4.],
        [4., 4.]])
tensor([[1., 1.],
        [1., 1.]])

Jacobian vector product

x = torch.randn(3, requires_grad=True)
print(x)

y = x * 2
i = 0
while y.data.norm() < 1000:
    y = y * 2
    i = i + 1
print(y)
print(i)

tensor([-0.9332,  1.9616,  0.1739], requires_grad=True)
tensor([-477.7843, 1004.3264,   89.0424], grad_fn=<MulBackward0>)
8

3. Parallel Computing
1) Why perform parallel computing—can calculate; calculate quickly; effect is good
2) How to perform parallel computing—CUDA
a. Distribute the network structure to different devices
b. Distribute the tasks of the same layer to different devices
c. Distribute different data to different devices

4. Resources and References

1. Feeding from Datawhale

Datawhale official tutorial online tutorial link for this course : https://datawhalechina.github.io/thorough-pytorch/
Github online tutorial: https://github.com/datawhalechina/thorough-pytorch
Gitee online tutorial: https://gitee. Com/datawhalechina/thorough-pytorch
B station video: https://www.bilibili.com/video/BV1L44y1472Z (Welcome everyone to click three links + pay attention!)

2. Feeding from the official

PyTorch's Github Portal:
https://github.com/pytorch/pytorch

3. Feeds from netizens

PyTorch detailed introduction
https://zhuanlan.zhihu.com/p/66543791
PyTorch Chinese tutorial
https://pytorch.apachecn.org/#/
Anaconda installation detailed tutorial
https://blog.csdn.net/in546/article/ details/117400839
CUDA installation detailed tutorial
https://blog.csdn.net/weixin_43848614/article/details/117221384
PyTorch installation is too slow Solution:
https://blog.csdn.net/Guoqi1911/article/details/110356637
Switch Tsinghua University Source detailed tutorial:
https://blog.csdn.net/lyj223061/article/details/108639378

---

Summarize

1. Thanks to the friends of Datawhale, everyone has organized this team study very seriously, and all the arrangements so far have been proper, reasonable and meticulous. With your help, I am full of motivation.
2. In the process of installing the environment, many problems will arise due to system differences, which is why many students give up immediately after getting started. In the process of solving problems, you must constantly ask for help, such as experience posts, bosses, teaching videos, etc. A little more patience will usher in success.
3. Before this PyTorch system study, I have actually installed PyTorch many times, and practiced some deep learning networks in some teaching projects. But since I never figured out the connection between the various software, I blindly installed and operated it according to the teaching posts, so there were many obstacles until this installation, let alone a deeper understanding of the network itself. Therefore, in the future, you should understand the details of the process, and not just blindly pursue the results. I believe that through this study, you can have a little bit of ability to answer questions for other students.