[Deep Learning] Pytorch Series Tutorials (1): PyTorch Data Structure: 1. Tensor (Tensor) and its Dimensions (Dimensions), Data Types (Data Types)

I. Introduction

ChatGPT：

  PyTorch is an open source machine learning framework that is widely used in the field of deep learning. It provides a rich set of tools and libraries for building and training various types of neural network models. Here is some detailed introduction to PyTorch:

• Dynamic calculation graph: PyTorch uses dynamic calculation graph for calculation, which means that the calculation graph can be dynamically defined, modified and adjusted at runtime, making the construction of the model and Debugging is more flexible and intuitive.
• Powerful GPU acceleration support: PyTorch makes full use of the GPU for calculations, which can greatly improve the speed of training and inference. It provides optimized operations and memory management for GPU, making running models on GPU more efficient.
• Automatic derivation: PyTorch has a built-in automatic derivation function, which can automatically calculate the gradient according to the defined calculation graph. This simplifies the implementation of the backpropagation algorithm and makes training neural network models more convenient.
• A large number of pre-trained models and model libraries: There are many pre-trained models and model libraries available in the PyTorch ecosystem, such as TorchVision, TorchText and TorchAudio, etc., which can be conveniently Load and use these models quickly, speeding up model development.
• High-level abstract interface: PyTorch provides high-level abstract interfaces, such as nn.Module and nn.functional, for quickly building neural network models. These interfaces encapsulate commonly used neural network layers and functions, simplifying the model definition and training process.
• Support distributed training: PyTorch supports distributed training on multiple GPUs and multiple machines, which can accelerate the training process and process large-scale data and models.
    Overall, PyTorch provides a flexible and powerful platform, making the research and development of deep learning more convenient and efficient. Its simple API and rich functions allow users to quickly implement complex neural network models and achieve excellent performance in various tasks.

2. Experimental environment

  This series of experiments uses the following environment

conda create -n DeepL python==3.11

conda activate DeepL

conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia

pip install matplotlib

  ˆ Regarding configuration environment issues, you can refer to the previous painful experience:

Anaconda builds deep learning environment py 3.7: tensorflow-gpu2.3.0, pytorch1.12.1_gpu version; (use conda to download cuda and cudnn); configuration environment experience summary

3. PyTorch data structure

0. Classification

• Tensor (Tensor): Tensor is the most basic data structure in PyTorch, similar to a multi-dimensional array. It can represent a scalar, vector, matrix, or array of arbitrary dimensions.
• Tensor operations: PyTorch provides a wealth of operation functions for performing various operations on Tensor, such as mathematical operations, statistical calculations, tensor deformation, indexing and slicing, etc. . These operation functions can efficiently utilize the GPU for parallel computing and accelerate the model training process.
• Variable (variable): Variable is an encapsulation of Tensor and is used for automatic derivation. In PyTorch, Variables automatically track and record operations performed on them, building computational graphs and supporting automatic derivation. In PyTorch 0.4.0 and later versions, Variable is abandoned, and Tensor can be used directly for automatic derivation.
• Dataset (data set): Dataset is an abstract class used to represent data sets. By inheriting the Dataset class, you can customize the data set and implement functions such as data loading, preprocessing, and sample acquisition. PyTorch also provides some built-in data set classes, such as MNIST, CIFAR-10, etc., to easily load commonly used data sets.
• DataLoader (data loader: DataLoader is used to load the data in the Dataset in batches and provides multi-thread and multi-process data pre-reading functions. It can efficiently Load large-scale data sets and support operations such as random shuffling of data, parallel loading, and data enhancement.
• Module (module): Module is the base class used to build models in PyTorch. By inheriting the Module class, you can define your own model and implement methods such as forward propagation and back propagation. Module provides functions such as parameter management, model saving and loading, etc. to facilitate model training and deployment.

1. Tensor

  Tensor (Tensor) is the main data structure used to represent multi-dimensional data in PyTorch. It is similar to a multi-dimensional array and can store and operate numeric data.

1. Dimensions

  The dimensions of a Tensor refer to the number of axes or order of the tensor. In PyTorch, you can use the size() method to obtain the dimension information of a tensor, and the dim() method to obtain the number of axes of a tensor.

0 dimensions (scalar)

import torch

# 创建0维张量（标量）
scalar = torch.tensor(5)
print("0维张量（标量）:")
print(scalar)
print("维度信息:", scalar.size())
print("轴数:", scalar.dim())

1 dimension (vector)

import torch

# 创建1维张量（向量）
vector = torch.tensor([1, 2, 3, 4, 5])
print("1维张量（向量）:")
print(vector)
print("维度信息:", vector.size())
print("轴数:", vector.dim())

2 dimensions (matrix)

import torch

# 创建2维张量（矩阵）
matrix = torch.tensor([[1, 2, 3], [4, 5, 6]])
print("2维张量（矩阵）:")
print(matrix)
print("维度信息:", matrix.size())
print("轴数:", matrix.dim())

3D tensor

import torch

# 创建3维张量
tensor_3d = torch.tensor([[[1, 2, 3], [4, 5, 6]], [[7, 8, 9], [10, 11, 12]]])
print("维张量:")
print(tensor_3d)
print("维度信息:", tensor_3d.size())
print("轴数:", tensor_3d.dim())

  In the above code, a 3-dimensional tensor tensor is created, which has 2 matrices with dimensions of 2x3. By calling the size() method, we can obtain the dimensional information of the tensor. What is returned is a torch.Size object, which is a data structure in the form of a tuple, representing the size of each dimension. In this example, the dimension information of tensor is [2, 2, 3], which means there are 2 matrices, each of which is 2x3. By calling the dim() method, we can get the number of axes of the tensor, which is 3 here.

2. Data Types

  Tensors in PyTorch can have different data types:

• torch.float32 or torch.float: 32-bit floating point tensor.
• torch.float64 or torch.double: 64-bit floating point tensor.
• torch.float16 or torch.half: 16-bit floating point tensor.
• torch.int8: 8-bit integer tensor.
• torch.int16 or torch.short: 16-bit integer tensor.
• torch.int32 or torch.int: 32-bit integer tensor.
• torch.int64 or torch.long: 64-bit integer tensor.
• torch.bool: Boolean tensor, storing True or False.

  When creating a tensor, you can set the required data type by specifying the dtype parameter. For example, to create a 64-bit floating point tensor, you can use the following code:

import torch

tensor = torch.tensor([1, 2, 3], dtype=torch.float64)
print(tensor)
print(tensor.dtype)