About pytorch tensor dimension conversion and tensor operations

tensor 乘
tensor 加
# view()    转换维度
# reshape() 转换维度
# permute() 坐标系变换
# squeeze()/unsqueeze() 降维/升维
# expand()   扩张张量
# narraw()   缩小张量
# resize_()  重设尺寸
# repeat(), unfold() 重复张量
# cat(), stack()     拼接张量

1 tensor.view()

view() is used to change the shape of the tensor , but does not change the element values ​​in the tensor .
Usage 1:
For example, you can use view to transform a tensor with shape (2, 3) into a tensor with shape (3, 2);

import torch
x = torch.tensor([[1, 2, 3], [4, 5, 6]])
y = x.view(3, 2)    

The above operation is equivalent to first flattening the tensor with shape **(2, 3) into (1, 6), and then into (3, 2).**

Usage 2:
The number of elements in the tensor remains unchanged before and after conversion. If the dimension of a certain dimension in view() is -1 , it means that the dimension of this dimension is adaptively adjusted according to the total number of elements and the size of other dimensions . Note that the dimension of at most one dimension in view() can be set to -1 .

z = x.view(-1,2)

For example:
In convolutional neural networks, views are often used in the fully connected layer to stretch the tensor dimensions:
Assume that the input feature is a 4-dimensional tensor of B C H*W , where B represents batchsize and C represents features. The number of channels, H and W represent the height and width of the feature. Before sending the feature to the fully connected layer, .view will be used to convert it into a 2-dimensional tensor of B*(C H W) , that is, the batch will remain unchanged, but Convert each feature into a one-dimensional vector.

2 tensor.reshape()

reshape() is used in the same way as view().

3 tensor.squeeze()和tensor.unsqueeze()

3.1 tensor.squeeze() dimensionality reduction

(1) If the squeeze() brackets are empty, all dimensions with a dimension of 1 in the tensor will be compressed , such as reducing the dimension of the tensor of 1, 2, 1, 9 to 2, 9 dimensions; if there is no 1 in the dimension If the dimension of the tensor is the same, the source dimension will remain unchanged. For example, if a 2 3 4-dimensional tensor is squeezed, the dimension will not change after conversion.
(2) If squeeze(idx) is used , the corresponding idx-th dimension in the tensor will be compressed. For example, if squeeze(2) is performed on tensors of 1, 2, 1, and 9, the dimension will be reduced to 1, 2, and 9 dimensions. Tensor; if the dimension of idx dimension is not 1, the dimension will not change after squeeze.
For example:

3.2 tensor.unsqueeze(idx)升维

Dimension upgrading is performed in the idx dimension, and the tensor is upgraded from the original dimension n to n+1 dimension . For example, the dimension of a tensor is 2*3. After unsqueeze(0), it becomes a tensor with dimensions of 1, 2, and 3.

4 tensor.permute()

Coordinate system transformation, that is, matrix transposition , is used in the same way as transpose of numpy array . The parameter numbers in permute() brackets refer to the index values ​​of each dimension. Permute is a technique often used in deep learning. Generally, the feature tensor of BCHW is converted into the feature tensor of BHWC through transposition , that is, the feature depth is converted to the last dimension by calling **tensor.permute(0 , 2, 3, 1)**realized.
torch.transpose can only operate the transpose of a 2D matrix, while the permute() function can transpose any high-dimensional matrix;
simple understanding: permute() is equivalent to operating several dimensions of tensor at the same time, and transpose can only act on tensor at the same time. of two dimensions.

Although both permute and view/reshape can convert tensors into specific dimensions, their principles are completely different, so pay attention to the distinction. After view and reshape processing, the order of elements in the tensor will not change, but the arrangement of elements will change after permute transposition because the coordinate system changes.

5 torch.cat([a,b],dim)

When performing tensor splicing in the dim dimension , attention should be paid to keeping the dimensions consistent .
Suppose a is a two-dimensional tensor of h1 w1, b is a two-dimensional tensor of h2 w2, torch.cat(a,b,0) means splicing in the first dimension , that is, splicing in the column direction , so w1 and w2 must equal. torch.cat(a,b,1) means splicing in the second dimension, that is, splicing in the row direction, so h1 and h2 must be equal .
Suppose a is a two-dimensional tensor of c1 h1 w1, and b is a two-dimensional tensor of c2 h2 w2. torch.cat(a,b,0) means splicing in the first dimension, that is, splicing in the channel dimension of the feature, Other dimensions must remain consistent, that is, w1=w2, h1=h2. torch.cat(a,b,1) means splicing in the second dimension, that is, splicing in the column direction. It must be ensured that w1=w2, c1=c2; torch.cat(a,b,2) means splicing in the third dimension. , that is, when splicing in the row direction, h1=h2, c1=c2 must be ensured;

6 torch.stack()

This function connects several tensors with the same shape in a dimension, and the final result will be dimensionally increased; that is, several tensors are connected in a certain dimension to generate an expanded tensor. Stacked feeling.

7 torch.chunk()和torch.split()

torch.chunk(input, chunks, dim)

The function of **torch.chunk()** is to evenly divide a tensor into several small tensors. Input is the divided tensor. Chunks is the number of evenly divided parts. If the size in the dimension of division is not divisible by chunks, the last tensor will be slightly smaller (or may be empty). dim determines the division along a certain dimension. This function returns a tuple composed of small tensors.

**torch.split()** can be said to be an upgraded version of torch.chunk(). It can not only split evenly according to the number of copies, but also split according to a specific plan.

torch.split(input, split_size_or_sections, dim=0)

The difference from torch.chunk() lies in the second parameter. If the second parameter is the number of divisions, this is the same as torch.chunk(); the second parameter is the division plan, which is a list type data. The tensor to be divided will be divided into len (list) parts. , the size of each portion depends on the elements in the list.

8 Multiplication operation with tensor

• Element-wise , that is, the corresponding elements of matrices of the same shape are multiplied. The obtained elements are the values ​​of each element in the result matrix. The corresponding function is torch.mul() (the same effect as * ).

• For matrix multiplication, the corresponding function is torch.mm() ( can only be used for 2D tensor) or torch.matmul() (same effect as the symbol @). For torch.matmul() , define its matrix multiplication only in the last two dimensions , and the previous dimensions need to be consistent. If the previous dimensions comply with the broadcast_tensor mechanism, the dimensions will be automatically expanded to ensure that the previous dimensions of the two matrices are consistent.

9 Addition operation with tensor

Follow these two points:

• When the dimensions of two tensors are the same , the values ​​of the corresponding axes should be the same (the size of each dimension is equal), or the size of some dimensions should be 1. When adding, copy and expand all the axes that are 1 to obtain two tensors with the same dimensions, and then add the corresponding positions.
• When the dimensions of the two added tensors are inconsistent , first align the tensor with a lower dimension from the right to the tensor with a higher dimension, expand the dimension with 1 to be consistent with the dimension of the high-dimensional tensor, and then perform <1> operate.

10 tensor.expand()

Expand a tensor to expand a single dimension to a larger size through value copying . Using the expand() function will not change the original tensor, and the result needs to be reassigned. The following are specific examples:
Take a two-dimensional tensor as an example: tensor is a 1 n or n 1-dimensional tensor. Call tensor.expand(s, n) or tensor.expand(n, s) respectively in the row direction and column direction. direction to expand.
The fill-in parameter of expand() is size

11 tensor.narrow(dim, start, len)

The narrow() function plays a role in filtering data in a certain dimension.

torch.narrow(input, dim, start, length)->Tensor

input is the tensor that needs to be sliced, dim is the slice dimension, start is the starting index, and length is the slice length. The actual application is as follows:

12 tensor.resize_()

Size changes, truncate the tensor to the dimensions after resize_.

13 tensor.repeat()

tensor.repeat(a,b) copies the entire tensor a copy in the row direction and b copies in the column direction.

14 unbind()

After torch.unbind() removes the specified dimension, it returns a tuple containing each slice along the specified dimension.

torch.unbind(input, dim=0)->seq

reference:

Functions related to tensor dimension changes in pytorch (continuously updated) - weili21's article - Zhihu
https://zhuanlan.zhihu.com/p/438099006

[pytorch tensor tensor dimension conversion (tensor dimension conversion)]
https://blog.csdn.net/x_yan033/article/details/104965077