PyTorch can be said to be the most suitable for beginners to learn among the three mainstream frameworks. Compared with other mainstream frameworks, PyTorch's simplicity and ease of use make it the first choice for beginners. The point I want to emphasize is that the framework can be compared to a programming language, which is only a tool for us to achieve project effects, that is, the wheels we use to build cars. What we need to focus on is to understand how to use Torch to implement functions without overly caring about it. How to make the wheels will take us too much learning time. In the future, there will be a series of articles that explain the deep learning framework in detail, but it is only later that we are more familiar with the theoretical knowledge and practical operations of deep learning before we can start learning. What we need most at this stage is to learn how to use these tools.

The content of deep learning is not so easy to master. It contains a lot of mathematical theoretical knowledge and a lot of calculation formula principles that require reasoning. And without actual operation, it is difficult to understand what role the code we write ultimately represents in the neural network computing framework. However, I will try my best to simplify the knowledge and convert it into content that we are more familiar with. I will try my best to let everyone understand and become familiar with the neural network framework, to ensure smooth understanding and smooth deduction, and try not to use too many mathematical formulas and Professional theoretical knowledge. Quickly understand and implement the algorithm in one article, and become proficient in this knowledge in the most efficient way.

The blogger has been focusing on data modeling for four years, and has participated in dozens of mathematical modeling, large and small, and understands the principles of various models, the modeling process of each model, and various problem analysis methods. The purpose of this column is to quickly use various mathematical models, machine learning, deep learning, and code from scratch. Each article contains practical projects and runnable code. Bloggers keep up with various digital and analog competitions. For each digital and analog competition, bloggers will write the latest ideas and codes into this column, as well as detailed ideas and complete codes. I hope friends in need will not miss the column carefully created by the author.

Quick Learning in One Article - Commonly Used Models in Mathematical Modeling

1. PyTorch data structure-Tensor

1.What is Tensor

Tensor is the most basic data structure in PyTorch and can be regarded as a multi-dimensional array (an extension of a matrix). Similar to arrays in NumPy, the difference is that ndarrays cannot use GPU to accelerate calculations but Tensor can run on GPU, which makes it particularly powerful in the field of deep learning. In the final analysis, we still have to rely on huge computing power to support us in order to get as much more information as possible Precise data is usually called a tensor.

2. Data Tensor usage scenarios

Before the concept of tensors was introduced, everyone had actually come into contact with these data in many scenarios. After understanding tensors, everyone will naturally understand. The core concept of a tensor is that it is a data container. The data it contains is almost always numerical data, so it is a container for numbers. You may be familiar with matrices, which are two-dimensional tensors. Tensors are a generalization of matrices to arbitrary dimensions (note that the dimension of a tensor is usually called the axis).

Here are some common usage scenarios:

Construction and training of neural networks : The input, weight, output, gradient, etc. of the neural network are all expressed in the form of Tensor. Deep learning frameworks such as PyTorch and TensorFlow all operate based on Tensor.
Image processing : Images are usually represented in the form of multi-dimensional arrays. For example, an RGB image can be represented as a Tensor of height, width and 3 color channels.
Natural language processing : When processing text data, the text can be encoded into a Tensor, and each word can be mapped to a number or a word vector.
Time series analysis : For example, stock prices, weather data, etc. can be expressed as Tensors in time and feature dimensions.
Recommendation system : The user-item interaction matrix can be represented as a two-dimensional Tensor.
Reinforcement learning : In algorithms such as Q-learning, information such as status, action, reward, etc. can be represented by Tensor.
Computer vision tasks : such as target detection, segmentation, face recognition and other tasks, input images, feature maps, target frame coordinates, etc. can all be represented by Tensor. The video is a 5D tensor with the shape of (samples, frames, height, width, channels) or (samples, frames, channels, height, width), and the picture 4D tensor has the shape of (samples, height, width, channels) or (samples , channels,height,width).
Autonomous driving and robot control : Sensor data (such as cameras, lidar, etc.) can be represented as Tensor for training and real-time control.
Generative Adversarial Network (GAN) : The generator and discriminator in GAN are trained and inferred based on neural networks and Tensors.
Clustering and dimensionality reduction : In data mining and feature engineering, Tensor can be used in algorithms such as clustering and principal component analysis (PCA).

In general, Tensor, as a multi-dimensional array, is suitable for almost all fields involving large-scale numerical computing and deep learning.

Here we use numpy to give you a more intuitive understanding of tensors.

3. Tensor shape

Scalar (0D tensor)

A tensor that contains only a single number is called a scalar, also called a scalar tensor, a zero-dimensional tensor, or an 0D tensor. In NumPy, a float32 or float64 number is a scalar tensor (or scalar array). You can find out the number of axes of a NumPy tensor by looking at its ndim property. A scalar tensor has 0 axes (ndim == 0). The number of axes is also called rank. The following is an example of a NumPy scalar.

import numpy as np
x = np.array(1)
print("数组中轴的个数:{}".format(x.ndim))
print(x)

Number of axes in array: 0 
1

vector (1D tensor)

An array of numbers is called a vector or one-dimensional tensor (1D tensor). A one-dimensional tensor has only one axis. Below is a Numpy vector.

x = np.array([1, 4, 7, 10, 13])
print("数组中轴的个数:{}".format(x.ndim))
print(x)

Number of axes in the array: 1 
[ 1 4 7 10 13 ]

It should be noted here that this array becomes a 5D vector but not a 5D tensor. The former is the quantity in quantity, and the latter is the number of axes. Don't get confused.

matrix (2D tensor)

An array of vectors is called a matrix or two-dimensional tensor (2D tensor). A matrix has 2 axes (often called rows and columns). You can intuitively think of a matrix as a rectangular grid of numbers. Below is a Numpy matrix.

x = np.array([[1, 8, 21, 3, 78],
              [8, 12, 13, 32, 11],
              [7, 3, 5, 6, 2]])
print("数组中轴的个数:{}".format(x.ndim))
print(x)

Number of axes in the array: 2 
[[ 1 8 21 3 78] 
 [ 8 12 13 32 11] 
 [ 7 3 5 6 2]]

3D tensors and high-dimensional tensors

Combining multiple matrices into a new array results in a 3D tensor, which you can intuitively understand as a cube of numbers. Below is a Numpy 3D tensor.

x = np.array([[[5, 78, 2, 34, 0],
               [6, 79, 3, 35, 1],
               [7, 80, 4, 36, 2]],
              
              [[5, 78, 2, 34, 0],
               [6, 79, 3, 35, 1],
               [7, 80, 4, 36, 2]],
        
              [[5, 78, 2, 34, 0],
               [6, 79, 3, 35, 1],
               [7, 80, 4, 36, 2]]])
print(x.ndim)
print(x)

3
[[[ 5 78  2 34  0]
  [ 6 79  3 35  1]
  [ 7 80  4 36  2]]

 [[ 5 78  2 34  0]
  [ 6 79  3 35  1]
  [ 7 80  4 36  2]]

 [[ 5 78  2 34  0]
  [ 6 79  3 35  1]
  [ 7 80  4 36  2]]]

In the future, when we come into contact with the corresponding data for tensor conversion, everyone will understand it more clearly. Now that we know the basic concepts and forms of tensors, we can try to use Torch to create them.

2. Creation of Tensor

In PyTorch, we can create Tensors in multiple ways:

1. Create from a list or NumPy array

import torch
import numpy as np

# 从列表创建
tensor_from_list = torch.Tensor([1, 2, 3, 4])

# 从NumPy数组创建
numpy_array = np.array([1, 2, 3, 4])
tensor_from_numpy = torch.Tensor(numpy_array)
tensor_from_numpy

tensor([1., 2., 3., 4.])

2. Use specific initialization methods

# 创建一个形状为(3, 2)的随机Tensor
random_tensor = torch.randn(3, 2)

# 创建一个全零的Tensor
zero_tensor = torch.zeros(3, 2)

# 创建一个全1的Tensor
ones_tensor = torch.ones(3, 2)
# 创建一个从 0 到 9 的等差数列
tensor5 = torch.arange(0, 10, 1)
# 创建一个从 0 到 1 的 5 个数的均匀间隔的数列
tensor6 = torch.linspace(0, 1, 5)
print(tensor6)
print(random_tensor)
print(zero_tensor)
print(ones_tensor)
print(tensor5)
print(tensor6)

tensor([0.0000, 0.2500, 0.5000, 0.7500, 1.0000])
tensor([[ 0.9255,  1.3031],
        [-1.2245, -0.5758],
        [-1.2821, -0.3471]])
tensor([[0., 0.],
        [0., 0.],
        [0., 0.]])
tensor([[1., 1.],
        [1., 1.],
        [1., 1.]])
tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
tensor([0.0000, 0.2500, 0.5000, 0.7500, 1.0000])

3. Create from existing Tensor

# 从已有的Tensor创建新的Tensor（共享内存）
new_tensor = torch.Tensor(random_tensor.size())
print(new_tensor)

tensor([[0., 0.],
        [0., 0.],
        [0., 1.]])

3. Tensor shape operations

1. View size and dimensions

There are two functions size() and shape that can be viewed:

random_tensor.size()
#random_tensor.shape 二者结果一致

torch.Size([3, 2])

2. Change the shape of Tensor

# 创建一个大小为 (2, 3) 的 Tensor
tensor = torch.arange(6).view(2, 3)
# 或者使用 reshape
# tensor = torch.arange(6).reshape(2, 3)
print(tensor)

tensor([[0, 1, 2],
        [3, 4, 5]])

3. Remove and increase the dimensions of Tensor

# squeeze 用于移除大小为 1 的维度
tensor = torch.arange(6).view(1, 2, 3)
squeezed_tensor = tensor.squeeze()
print(squeezed_tensor)

# unsqueeze 用于增加维度
tensor = torch.arange(6).view(2, 3)
unsqueeze_tensor = tensor.unsqueeze(0)
print(unsqueeze_tensor)

tensor([[0, 1, 2],
        [3, 4, 5]])
tensor([[[0, 1, 2],
         [3, 4, 5]]])

4. Exchange Tensor dimensions

# 交换维度
tensor = torch.arange(6).view(2, 3)
transposed_tensor = tensor.transpose(0, 1)  # 交换维度 0 和 1
print(transposed_tensor)

tensor([[0, 3],
        [1, 4],
        [2, 5]])

5. Splicing and stacking Tensors

# cat 用于在指定维度上连接多个 Tensor
tensor1 = torch.arange(3)
tensor2 = torch.arange(3, 6)
concatenated_tensor = torch.cat((tensor1, tensor2), dim=0)
print(concatenated_tensor)

# stack 用于在新的维度上堆叠多个 Tensor
stacked_tensor = torch.stack((tensor1, tensor2), dim=0)
print(stacked_tensor)

6. Split Tensor

Original Tensor:

tensor = torch.arange(10).view(2, 5)
print(tensor)

Split results

# split 用于在指定维度上拆分 Tensor
tensor = torch.arange(10).view(2, 5)
split_tensors = torch.split(tensor, 2, dim=1)
print(split_tensors)

# chunk 用于在指定维度上将 Tensor 拆分成相等的几块
chunked_tensors = torch.chunk(tensor, 2, dim=0)
print(chunked_tensors)

(tensor([[0, 1],
        [5, 6]]), tensor([[2, 3],
        [7, 8]]), tensor([[4],
        [9]]))
(tensor([[0, 1, 2, 3, 4]]), tensor([[5, 6, 7, 8, 9]]))

4. Tensor indexing and slicing

1.Index

import torch

# 创建一个二维 Tensor
tensor = torch.Tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

# 访问单个元素
print(tensor[0, 0])  # 输出 1

# 访问某一行
print(tensor[1])  # 输出 [4, 5, 6]

# 访问某一列
print(tensor[:, 1])  # 输出 [2, 5, 8]

# 使用逗号分隔访问多个不连续的元素
print(tensor[0, 0], tensor[1, 1], tensor[2, 2])  # 输出 1, 5, 9

tensor(1.)
tensor([4., 5., 6.])
tensor([2., 5., 8.])
tensor(1.) tensor(5.) tensor(9.)

2. Slice

# 使用切片获取子张量
print(tensor[0:2, 1:3])  # 输出 [[2, 3], [5, 6]]

# 使用步长获取间隔元素
print(tensor[::2])  # 输出 [[1, 2, 3], [7, 8, 9]]

# 使用负数索引倒序访问
print(tensor[-1])  # 输出 [7, 8, 9]

tensor([[2., 3.],
        [5., 6.]])
tensor([[1., 2., 3.],
        [7., 8., 9.]])
tensor([7., 8., 9.])

3. Modify Tensor through indexing and slicing

(1). Modify the value of a single element

# 创建一个二维 Tensor
tensor = torch.Tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
# 修改单个元素的值
tensor[0, 0] = 0
tensor

tensor([[0., 2., 3.],
        [4., 5., 6.],
        [7., 8., 9.]])

(2). Modify the entire row or column

Whole line modification:

# 创建一个二维 Tensor
tensor = torch.Tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
# 修改整行或整列
tensor[1] = torch.Tensor([10, 11, 12])
tensor

Alignment fix:

# 创建一个二维 Tensor
tensor = torch.Tensor([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
# 修改整行或整列
tensor[:, 2] = torch.Tensor([30, 31, 32])
tensor

tensor([[ 1.,  2., 30.],
        [ 4.,  5., 31.],
        [ 7.,  8., 32.]])

4. Advanced indexing

(1).Use a list as an index

idx = [0, 2]
print(tensor[idx])  # 输出第 0 和第 2 行

tensor([[ 1.,  2., 30.],
        [ 7.,  8., 32.]])

(2).Use Boolean value as index (mask)

mask = tensor > 5
print(tensor[mask])  # 输出所有大于 5 的元素

tensor([30., 31.,  7.,  8., 32.])

(3).Use torch’s index function

# 使用 torch 的索引函数
idx = torch.LongTensor([0, 2])
print(torch.index_select(tensor, 0, idx))  # 输出第 0 和第 2 行

tensor([[ 1.,  2., 30.],
        [ 7.,  8., 32.]])

Please pay attention to prevent it from getting lost. If there are any mistakes, please leave a message for advice. Thank you very much.

That’s all for this issue. My name is fanstuck. If you have any questions, feel free to leave a message for discussion. See you in the next issue.

PyTorch actual combat-detailed explanation of the most complete operation of Tensor, the basis of neural network image classification (1)

Preface