Detailed explanation of PyTorch Tensor data type - tensor

Table of contents

tensor

tensor structure

tensor creation

tensor operations

computation

logic operation

shape manipulation

Indexing and Slicing

Dimensional Transformation of Tensors

Common Data Type Storage

Appendix PyTorch Official API Chinese Documentation

---

tensor

In PyTorch, tensor (Tensor) is a common data type, which is a multidimensional array. Unlike lists in Python or arrays in NumPy, tensors can run on the GPU, which makes them very useful in deep learning. This article will introduce the structure and usage of tensors, and how to use them in PyTorch.

tensor structure

Tensors are described by a data type and a shape. shape is a tuple containing the size of each dimension. For example, a tensor of shape (2, 3, 4) represents a three-dimensional array where the first dimension has size 2, the second dimension has size 3, and the third dimension has size 4. Here's a simple example that creates a tensor of shape (2, 3, 4):

import torch

# 创建一个形状为(2, 3, 4)的张量
x = torch.randn(2, 3, 4)
print(x)
print(x.shape)

The above code outputs the following:

 

In the above code, torch.randnit is a function to create a random tensor, which uses the given shape to create a random tensor of shape (2, 3, 4). We use print(x)the output tensor, using print(x.shape)the shape of the output tensor.

In addition to shape, each tensor also has a data type. PyTorch supports a variety of data types, including floats, integers, and booleans. The following are some common data types:

• torch.float32or torch.float: 32-bit float
• torch.float64or torch.double: 64-bit float
• torch.int8: 8-bit integer
• torch.int16or torch.short: 16-bit integer
• torch.int32or torch.int: 32-bit integer
• torch.int64or torch.long: 64-bit integer
• torch.bool:Boolean value

tensor creation

The basic structure of a tensor is a multidimensional array where each element has the same data type and shape. In PyTorch, we can use torch.tensorfunctions to create tensors and specify its shape and data type. float32For example, we can use the following code to create a tensor of shape (3, 2) and data type :

import torch

# 创建一个形状为(3, 2)、数据类型为float32的张量
x = torch.tensor([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]], dtype=torch.float32)

print(x)

The above code outputs the following:

In the above code, we use torch.tensorthe function to create a float32tensor with shape (3, 2) and data type x. Among them, dtypethe parameter is used to specify the data type of the tensor, which can be torch.float32, torch.float64, torch.int32, torch.int64etc.

In addition torch.tensorto functions, PyTorch also provides many other tensor creation functions, such as torch.zeros, torch.ones, torch.randnetc. These functions are used in torch.tensora similar fashion and detailed instructions can be found in PyTorch's documentation.

tensor operations

 In PyTorch, we can perform various operations on tensors, including mathematical operations, logical operations, and shape operations. Below are some common tensor operations.

computation

Tensors can perform various mathematical operations such as addition, subtraction, multiplication, division, exponentiation, etc. For example, we can use torch.addfunctions for tensor addition:

import torch

# 创建两个形状为(2, 3)的张量
x = torch.randn(2, 3)
y = torch.randn(2, 3)

# 张量相加
z = torch.add(x, y)
print(z)

In the above code, we created the sum of two tensors of shape (2, 3) xand ythen torch.addadded them using a function to get a tensor of the same shape z.

In addition torch.addto functions, PyTorch also provides many other mathematical operation functions, such as torch.sub, torch.mul, torch.div, torch.expand so on. These functions are used in torch.adda similar fashion and detailed instructions can be found in PyTorch's documentation.

logic operation

Tensors can also perform logical operations, such as comparison, logical and, logical or, etc. For example, we can use torch.eqfunctions for tensor comparisons:

import torch

# 创建两个形状为(2, 3)的张量
x = torch.tensor([[1, 2, 3], [4, 5, 6]])
y = torch.tensor([[4, 5, 6], [1, 2, 3]])

# 张量比较
z = torch.eq(x, y)
print(z)

 

In the above code, we created the sum of two tensors of shape (2, 3) xand compared them yusing torch.eqa function to get boolean tensors of the same shape z.

In addition torch.eqto functions, PyTorch also provides many other logical operation functions, such as torch.gt, torch.lt、torch.ge, torch.le, torch.neetc. These functions are used in torch.eqa similar fashion and detailed instructions can be found in PyTorch's documentation.

shape manipulation

Tensors can also undergo shape operations, including reshaping, concatenating, slicing, and more. For example, we can torch.reshapechange the shape of a tensor using a function:

import torch

# 创建一个形状为(2, 3)的张量
x = torch.randn(2, 3)

# 改变张量的形状
y = torch.reshape(x, (3, 2))
print(y)

In the above code, we created a tensor of shape (2, 3) xand then used torch.reshapea function to transform it into a tensor of shape (3, 2) y.

In addition torch.reshapeto functions, PyTorch also provides many other shape manipulation functions, such as torch.cat, torch.stack, torch.split, torch.chunketc. These functions are used in torch.reshapea similar fashion and detailed instructions can be found in PyTorch's documentation.

Indexing and Slicing

In PyTorch, we can use indexing and slicing operations to access elements in tensors. For example, we can access individual elements in a tensor using indexing operations:

import torch

# 创建一个形状为(2, 3)的张量
x = torch.tensor([[1, 2, 3], [4, 5, 6]])

# 使用索引操作访问张量中的单个元素
print(x[0, 1])

In the above code, we created a tensor with shape (2, 3) x, and then used the index operation to access its first row and second column elements.

In addition to indexing operations, PyTorch also supports various slicing operations, such as using colon (:) for slicing. For example, we can access a portion of a tensor using the slice operation:

import torch

# 创建一个形状为(2, 3)的张量
x = torch.tensor([[1, 2, 3], [4, 5, 6]])

# 使用切片操作访问张量的一部分
print(x[:, 1:])

 

 In the above code, we created a tensor of shape (2, 3) x, and then used the slice operation to access the second column and subsequent elements of all its rows.

Dimensional Transformation of Tensors

In PyTorch, we can use various tensor operations to change the dimensions of tensors in order to adapt to different tasks and models. Here are several commonly used tensor dimension transformation operations:

1. reshape: This operation can change the shape of the tensor, but does not change the number of elements in the tensor. For example, a tensor of shape (1, 3, 224, 224) can be converted to a tensor of shape (3, 224, 224), the code is as follows:

import torch

x = torch.randn(1, 3, 224, 224)
y = x.reshape(3, 224, 224)
print(y.shape)  # 输出：torch.Size([3, 224, 224])

2. Transpose: This operation can exchange the dimension order of tensors. For example, to convert a tensor with shape (1, 3, 224, 224) into a tensor with shape (224, 224, 3), the code is as follows:

import torch

x = torch.randn(1, 3, 224, 224)
y = x.transpose(1, 3).transpose(1, 2)
print(y.shape)  # 输出：torch.Size([224, 224, 3])

3. unsqueeze/unsqueeze: This operation can add a new dimension to the specified dimension, for example, convert a tensor with shape (3, 224, 224) into a tensor with shape (1, 3, 224, 224) amount, the code is as follows:

import torch

x = torch.randn(3, 224, 224)
y = x.unsqueeze(0)
print(y.shape)  # 输出：torch.Size([1, 3, 224, 224])

4. squeeze/unsqueeze: This operation can remove a dimension in the specified dimension, for example, convert a tensor with shape (1, 3, 224, 224) to a tensor with shape (3, 224, 224) ,code show as below:

import torch

x = torch.randn(1, 3, 224, 224)
y = x.squeeze(0)
print(y.shape)  # 输出：torch.Size([3, 224, 224])

In short, in PyTorch, the tensor dimension transformation operation can help us conveniently carry out data processing and model construction, and improve the readability and flexibility of the code. It is necessary to select the appropriate operation according to the specific needs in order to achieve the optimal effect.

Common Data Type Storage

In a computer, different types of data have their own specific storage and encoding methods. The following briefly introduces how common data types are stored in computers and represented in tensors:

1. String Literal: A string literal is a sequence of characters in a computer, each of which occupies one or more bytes. In PyTorch, a string literal can be represented as a 1-dimensional tensor, where each element represents the encoding of a character.

2. Picture: Pictures are usually stored in the form of a pixel matrix in the computer, and the color value of each pixel can be represented by different encoding methods, such as RGB, grayscale, CMYK, etc. In PyTorch, a picture can be represented as a 4-dimensional tensor, where the 0th dimension represents the number of batches, the 1st dimension represents the number of channels, the 2nd dimension represents the height, and the 3rd dimension represents the width. For example, a tensor of shape (1, 3, 224, 224) can represent a 224x224 image in RGB format.

3. Video: A video is composed of a series of frames, and each frame can be regarded as a picture. In computers, video is usually stored as a sequence of frames. In PyTorch, a video can be represented as a 5-dimensional tensor, where the 0th dimension represents the number of batches, the 1st dimension represents the number of frames, the 2nd dimension represents the number of channels, the 3rd dimension represents height, and the 4th dimension represents width. For example, a tensor of shape (1, 16, 3, 224, 224) could represent a 224x224 video in RGB format with 16 frames.

4. Sound: Sound is a continuous signal that can be represented using different sampling rates and encoding methods. In PyTorch, sound can be represented as a 1-dimensional tensor, where each element represents the value of a sample point. It should be noted that the sample rate and bit depth of the sound signal determine the size and data type of the tensor.

In short, different types of data have their own storage and encoding methods in computers, and in PyTorch, these data can be represented as tensors, and various tensor operations can be performed conveniently. In practical applications, we usually need to convert different types of data into tensors and feed them into neural networks for training and prediction.

Appendix PyTorch Official API Chinese Documentation

The Chinese version of PyTorch's official API documentation can be viewed at the following URL:

https://pytorch.apachecn.org/docs/1.9/