If you need source code files, please like and follow the collection and leave a private message in the comment area~~~
1. Tensor module
Tensor is the most basic operation object of PyTorch, and it is a multidimensional array with a uniform type. Everyone is very familiar with scalars, vectors and matrices, but when we want to describe a high-dimensional data, scalars, vectors and matrices are somewhat "incapable", so tensors came into being
In geometric definitions, tensors are extensions based on the concepts of scalars, vectors, and matrices. To understand it more generally, scalars can be regarded as 0-dimensional tensors, vectors as 1-dimensional tensors, and matrices as 2-dimensional tensors. In the field of deep learning, tensor can be regarded as a bucket of data. When only one drop of water is placed in the bucket, it is a 0-dimensional tensor. If multiple drops of water are arranged in a row, it is a 1-dimensional tensor, and if they are arranged in a plane, it is a 2-dimensional tensor. , and so on, extended to n-dimensional tensors
Like Python numbers and strings, all tensors are immutable: the contents of a tensor can never be updated, only new ones can be created
1: The data type of the tensor
create tensor
Generate random numbers
torch.rand(): Generate random numbers that obey the uniform distribution;
torch.randn(): Generate random numbers that obey the standard normal distribution;
torch.normal(): specifies the random number of the normal distribution of the mean and standard deviation;
torch.linspace(): Generate evenly spaced random numbers;
torch.manual_seed(): used to fix the random seed and generate the same random number;
torch.ones()、torch.zeros()、torch.eye()
The following is the code for the basic operation above
import torch
# 从python数组构建
a = [[1, 2, 3],[4, 5, 6]]
x = torch.Tensor(a)
print(a, x)
# 输出结果为：
# tensor([[1., 2., 3.],
# [4., 5., 6.]])
# 从列表构建张量
x = torch.Tensor([[1, 2]])
print(x)
# 输出结果为：
# tensor([[1., 2.]])
tensor1 = torch.rand(4)
tensor2 = torch.rand(2, 3)
print(tensor1, tensor2)
# tensor([0.7638, 0.3919, 0.9474, 0.6846])
# tensor([[0.3425, 0.0689, 0.6304],
# [0.5676, 0.8049, 0.3459]])
tensor1 = torch.randn(5)
tensor2 = torch.randn(2, 4)
print(tensor1, tensor2)
# tensor([ 0.4315, -0.3812, 0.9554, -0.8051, -0.9421])
# tensor([[-0.6991, 0.0359, 1.2298, -0.1711],
# [ 1.0056, 0.5772, 1.4460, -0.5936]])
tensor = torch.normal(mean=torch.arange(1., 11.), std= torch.arange(1, 0, -0.1))
print(tensor)
# tensor([0.0605, 2.5965, 3.3046, 4.2056, 5.0117, 6.7848, 6.3024, 7.9845, 9.4306, 9.7881])
torch.arange(1, 0, -0.1)
tensor = torch.normal(mean=0.5, std=torch.arange(1., 6.))
print(tensor)
# tensor([-0.0757, -0.5302, -1.1334, -4.3958, -5.8655])
tensor = torch.normal(mean=torch.arange(1., 6.), std=1.0)
print(tensor)
# tensor([1.6546, 2.7788, 2.4560, 3.2527, 4.1715])
tensor = torch.normal(2, 3, size=(1, 4))
print(tensor)
# tensor([[ 4.7555, -2.5026, -1.6333, -0.9256]])
tensor = torch.linspace(1, 10, steps=5)
print(tensor)
# tensor([ 1.0000, 3.2500, 5.5000, 7.7500, 10.0000])
torch.manual_seed(1)
temp1 = torch.rand(5)
print(temp1) # tensor([0.7576, 0.2793, 0.4031, 0.7347, 0.0293])
torch.manual_seed(1)
temp2 = torch.rand(5)
print(temp2) # tensor([0.7576, 0.2793, 0.4031, 0.7347, 0.0293])
temp3 = torch.rand(5)
print(temp3) # tensor([0.7999, 0.3971, 0.7544, 0.5695, 0.4388])
tensor1 = torch.zeros(2, 3)
tensor2 = torch.ones(2, 3)
tensor3 = torch.eye(3)
print(tensor1, tensor2, tensor3)
# tensor([[0., 0., 0.],
# [0., 0., 0.]])
# tensor([[1., 1., 1.],
# [1., 1., 1.]])
# tensor([[1., 0., 0.],
# [0., 1., 0.],
# [0., 0., 1.]])
# 第一种方法：在创建张量时指定数据类型
x = torch.ones((2, 3, 4), dtype=torch.int64) # 生成全1数组
print(x)
# 输出结果为：
# tensor([[[1, 1, 1, 1],
# [1, 1, 1, 1],
# [1, 1, 1, 1]],
#
# [[1, 1, 1, 1],
# [1, 1, 1, 1],
# [1, 1, 1, 1]]])
# 第二种方法：张量创建完成后，对数据类型进行转换
x = torch.ones(2, 3, 4) # 生成全1数组
x = x.type(torch.int64)
print(x)
# 输出结果为：
# tensor([[[1, 1, 1, 1],
# [1, 1, 1, 1],
# [1, 1, 1, 1]],
#
# [[1, 1, 1, 1],
# [1, 1, 1, 1],2: Basic operations of tensors
change the shape of a tensor
x = torch.rand(3, 2)
print(x.shape) # torch.Size([3, 2])
y = x.view(2, 3)
print(y.shape) # torch.Size([6])Add and remove dimensions
# 增加维度
a = torch.rand(3, 4)
b = torch.unsqueeze(a, 0)
c = a.unsqueeze(0)
print(b.shape) # torch.Size([1, 3, 4])
print(c.shape) # torch.Size([1, 3, 4])swap dimensions
# 删除维度
a = torch.rand(1, 1, 3, 4)
b = torch.squeeze(a)
c = a.squeeze(1)
print(b.shape) # torch.Size([3, 4])
print(c.shape) # torch.Size([1, 3, 4])Stitching and Segmentation
# torch.cat()拼接方法的代码如下：
a = torch.rand(1, 2)
b = torch.rand(1, 2)
c = torch.rand(1, 2)
output1 = torch.cat([a, b, c], dim=0) # dim=0为按列拼接
print(output1.shape) # torch.Size([3, 2])
output2 = torch.cat([a, b, c], dim=1) # dim=1为按行拼接
print(output2.shape) # torch.Size([1, 6])stacking and breaking down
# torch.stack()堆叠方法的代码如下：
a = torch.rand(1, 2)
b = torch.rand(1, 2)
c = torch.rand(1, 2)
output1 = torch.stack([a, b, c], dim=0) # dim=0为按列拼接
print(output1.shape) # torch.Size([3, 1, 2])
output2 = torch.stack([a, b, c], dim=1) # dim=1为按行拼接
print(output2.shape) # torch.Size([1, 3, 2])
# torch.chunk()分解方法的代码如下：
a = torch.rand(3, 4)
output1 = torch.chunk(a, 2, dim=0)
print(output1)
# (tensor([[0.1943, 0.1760, 0.3022, 0.0746],
# [0.5819, 0.7897, 0.2581, 0.0709]]), tensor([[0.2137, 0.5694, 0.1406, 0.0052]]))
output2 = torch.chunk(a, 2, dim=1)
print(output2)
# (tensor([[0.1943, 0.1760],
# [0.5819, 0.7897],
# [0.2137, 0.5694]]), tensor([[0.3022, 0.0746],
# [0.2581, 0.0709],
# [0.1406, 0.0052]]))Indexing and Slicing
x = torch.rand(2, 3, 4)
print(x[1].shape) # torch.Size([3, 4])
y = x[1, 0:2, :]
print(y.shape) # torch.Size([2, 4])
z = x[:, 0, ::2]
print(z.shape) # torch.Size([2, 2])Here is the basic math
Sum Element-wise Sum Element-wise Product by Index Mean Find Variance Max Find Min
# 元素求和第一种方法
a = torch.rand(4, 3)
b = torch.sum(a)
print(b) # tensor(6.4069)
# 元素求和第二种方法
a = torch.rand(4, 3)
b = torch.sum(a, dim=1, keepdim=False)
print(b, b.shape)
# tensor([[0.6594],
# [1.5325],
# [1.5375],
# [1.7755]]) torch.Size([4, 1])
# 按索引求和,不常用
x = torch.Tensor([[1, 2],[3, 4]])
y = torch.Tensor([[3, 4],[5, 6]])
index = torch.LongTensor([0, 1])
output = x.index_add_(0, index, y)
print(output)
# tensor([[ 4., 6.],
# [ 8., 10.]])
# 元素乘积第一种方法
a = torch.rand(4, 3)
b = torch.prod(a)
print(b) # tensor(2.0311e-05)
# 元素乘积第二种方法
a = torch.rand(4, 3)
b = torch.prod(a, 1, True)
print(b, b.shape)
# tensor([[0.0194],
# [0.1845],
# [0.0336],
# [0.4879]]) torch.Size([4, 1])
# 求平均数的第一种方法
a = torch.rand(4, 3)
b = torch.mean(a)
print(b) # tensor(0.4836)
# 求平均数的第二种方法
a = torch.rand(4, 3)
b = torch.mean(a, 1, True)
print(b, b.shape)
# tensor([[0.6966],
# [0.6087],
# [0.3842],
# [0.1749]]) torch.Size([4, 1])
# 求方差的第一种方法
a = torch.rand(4, 3)
b = torch.var(a)
print(b) # tensor(0.0740)
# 求方差的第二种方法
a = torch.rand(4, 3)
b = torch.var(a, 1, True)
print(b, b.shape)
# tensor([0.1155, 0.0874, 0.0354, 0.0005]) torch.Size([4, 1])
# 求最大值的第一种方法
a = torch.rand(4, 3)
b = torch.max(a)
print(b) # tensor(0.8765)
# 求最大值的第二种方法
a = torch.rand(4, 3)
b = torch.max(a, 1, True)
print(b)
# torch.return_types.max(
# values = tensor([[0.9875],
# [0.6657],
# [0.9412],
# [0.7775]]),
# indices = tensor([[2],
# [0],
# [0],
# [1]]))
# 求最小值的第一种方法
a = torch.rand(4,3)
b = torch.min(a)
print(b) # tensor(0.0397)
# 求最小值的第二种方法
a = torch.rand(4, 3)
b = torch.min(a, 1, True)
print(b)
# torch.return_types.min(
# values = tensor([[0.0436],
# [0.1586],
# [0.4904],
# [0.2536]]),
# indices = tensor([[0],
# [1],
# [2],
# [1]]))
Vector Operations and Matrix Operations
Including the dot product of vectors the cross product of vectors the inner product of matrices the outer product of matrices
# 向量的点乘,a和b必须为一维
a = torch.Tensor([1, 2, 3])
b = torch.Tensor([1, 1, 1])
output = torch.dot(a, b)
print(output) # 等价于 1*1+2*1+3*1，tensor(6.)
# 向量的叉乘
a = torch.Tensor([1, 2, 3])
b = torch.Tensor([1, 1, 1])
output = torch.multiply(a, b)
print(output) # tensor([1., 2., 3.])
# 矩阵的内积
a = torch.Tensor([1, 2, 3])
b = torch.Tensor([1, 1, 1])
output = torch.inner(a, b)
print(output) # tensor(6.)
# 矩阵的外积：矩阵乘法
a = torch.Tensor([[1, 2, 3], [4, 5, 6]])
b = torch.Tensor([[1, 1], [2, 2], [3, 3]])
output = torch.matmul(a, b)
print(output)
# tensor([[14., 14.],
# [32., 32.]])
# 按批量相乘
a = torch.randn(10, 3, 4)
b = torch.randn(10, 4, 5)
output = torch.bmm(a, b)
print(output.shape)
# tensor([[14., 14.],
# [32., 32.]])3: Tensors and Numpy arrays
Since it is very convenient to use ndarray in numpy to process data, tensors and numpy arrays are often converted to each other, so it is necessary to master the conversion method between the two
Tensor to numpy array: tensor.numpy()
Numpy array to tensor: torch.from_numpy()
Tensor to Numpy array
a = torch.ones(1, 2)
b = a.numpy() # 进行转换
print(a, b) # tensor([[1., 1.]]) [[1. 1.]]
a += 2
print(a, b) # tensor([[3., 3.]]) [[3. 3.]]
b += 2 # 在a改变后，b也已经改变
print(a, b) # tensor([[5., 5.]]) [[5. 5.]]Numpy array to tensor
import numpy as np
a = np.ones([1, 2])
b = torch.from_numpy(a) # 进行转换
print(a, b) # [[1. 1.]] tensor([[1., 1.]], dtype=torch.float64)
a += 2
print(a, b) # [[3. 3.]] tensor([[3., 3.]], dtype=torch.float64)
b += 2 # 在a改变后，b也已经改变
print(a, b) # [[5. 5.]] tensor([[5., 5.]], dtype=torch.float64)4: Cuda tensor and CPU tensor
In the process of deep learning, GPU can play an accelerated role. Tensors in Pytorch are stored in the CPU device by default. If the GPU is available, the tensor can be transferred to the GPU.
x = torch.rand(2, 4)
print(x.device) # cpu
# 第一种方法
x = x.cuda()
print(x.device) # cuda:0
# 第二种方法
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
if torch.cuda.is_available():
x = x.to(device)
print(x.device) #cuda:0
# 转化为cpu
x = x.cpu()
print(x.device) # cpuIt's not easy to create and find it helpful, please like, follow and collect~~~