Table of contents
I. Introduction
ChatGPT：
PyTorch is an open source machine learning framework that is widely used in the field of deep learning. It provides a rich set of tools and libraries for building and training various types of neural network models. Here is some detailed introduction to PyTorch:
Dynamic calculation graph: PyTorch uses a dynamic calculation graph for calculations, which means that the calculation graph can be dynamically defined, modified and adjusted at runtime, making model construction and debugging more flexible and intuitive.
Powerful GPU acceleration support: PyTorch makes full use of the GPU for calculations, which can greatly improve the speed of training and inference. It provides optimized operations and memory management for GPU, making running models on GPU more efficient.
Automatic derivation: PyTorch has a built-in automatic derivation function, which can automatically calculate the gradient according to the defined calculation graph. This simplifies the implementation of the backpropagation algorithm and makes training neural network models more convenient.
A large number of pre-trained models and model libraries: There are many pre-trained models and model libraries available in the PyTorch ecosystem, such as TorchVision, TorchText, and TorchAudio, etc. These models can be easily loaded and used to speed up model development.
High-level abstraction interface: PyTorch provides high-level abstraction interfaces, such as
nn.Moduleandnn.functional, for quickly building neural network models. These interfaces encapsulate commonly used neural network layers and functions, simplifying the model definition and training process.Support distributed training: PyTorch supports distributed training on multiple GPUs and multiple machines, which can accelerate the training process and process large-scale data and models.
Overall, PyTorch provides a flexible and powerful platform that makes deep learning research and development more convenient and efficient. Its simple API and rich functions allow users to quickly implement complex neural network models and achieve excellent performance in various tasks.
2. Experimental environment
This series of experiments uses the following environment
conda create -n DL python=3.7 conda activate DLpip install torch==1.8.1+cu102 torchvision==0.9.1+cu102 torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html
conda install matplotlibRegarding configuration environment issues, you can refer to the previous painful experience:
3. PyTorch data structure
0. Classification
- Tensor : Tensor is the most basic data structure in PyTorch, similar to a multi-dimensional array. It can represent a scalar, vector, matrix, or array of arbitrary dimensions.
- Tensor operations : PyTorch provides a wealth of operation functions for performing various operations on Tensor, such as mathematical operations, statistical calculations, tensor deformation, indexing and slicing, etc. These operation functions can efficiently utilize the GPU for parallel computing and accelerate the model training process.
- Variable : Variable is an encapsulation of Tensor and is used for automatic derivation. In PyTorch, Variables automatically track and record operations performed on them, building computational graphs and supporting automatic derivation. In PyTorch 0.4.0 and later versions, Variable is abandoned, and Tensor can be used directly for automatic derivation.
- Dataset : Dataset is an abstract class used to represent data sets. By inheriting the Dataset class, you can customize the data set and implement functions such as data loading, preprocessing, and sample acquisition. PyTorch also provides some built-in data set classes, such as MNIST, CIFAR-10, etc., to easily load commonly used data sets.
- DataLoader : DataLoader is used to load the data in the Dataset in batches and provides multi-thread and multi-process data pre-reading functions. It can efficiently load large-scale data sets and supports operations such as random shuffling of data, parallel loading, and data enhancement.
- Module : Module is the base class used to build models in PyTorch. By inheriting the Module class, you can define your own model and implement methods such as forward propagation and back propagation. Module provides functions such as parameter management, model saving and loading, etc. to facilitate model training and deployment.
1. Tensor _ _
https://blog.csdn.net/m0_63834988/article/details/1329092192. Tensor Operations
3. Variable
In PyTorch, Variable (variable) is a concept in earlier versions and is used for automatic differentiation (autograd). However, starting from PyTorch version 0.4.0, Variable has been deprecated, and the automatic derivation function is integrated directly into the tensor (Tensor), so there is no need to use Variable explicitly.
In earlier versions of PyTorch, Variable was a way to wrap a tensor, which contained the tensor's data, gradients, and other information related to automatic derivation. You can perform various operations on Variable, just like operating tensors, and it will automatically record gradient information. Then, by calling .backward()the method, you can backpropagate the Variable, calculate the gradient, and propagate the gradient to the relevant variables.
import torch
from torch.autograd import Variable
# 创建一个Variable
x = Variable(torch.tensor([2.0]), requires_grad=True)
# 定义一个计算图
y = x ** 2 + 3 * x + 1
# 进行反向传播
y.backward()
# 获取梯度
gradient = x.grad
print("梯度：", gradient) # 输出: tensor([7.]) In the above code, we first wrap the tensor torch.tensor([2.0])into a Variable and set it requires_grad=Trueto indicate that we want to calculate the gradient of this variable. Then, we defined a computational graph and calculated y = x ** 2 + 3 * x + 1. Next, we call y.backward()backpropagation on Variable to calculate the gradient. Finally, we x.gradget the gradient value via .
It should be noted that in PyTorch 0.4.0 and higher, Variable has been deprecated and automatic derivation is integrated directly into tensors. Therefore, you can directly perform automatic differentiation on tensor usage .backward()methods without explicitly using Variable.
import torch
x = torch.tensor([2.0], requires_grad=True)
# 定义一个计算图
y = x ** 2 + 3 * x + 1
# 进行反向传播
y.backward()
# 获取梯度
gradient = x.grad
print("梯度：", gradient) # 输出: tensor([7.])