Deep learning--tensor operations and mathematical operations on linear regression tensors and building linear regression models with tensors

foreword

This blog inherits the content of the previous blog, and will explain the operation of tensors. At the same time, on the basis of understanding some mathematics of tensors, and with the theory of machine learning, the construction of a linear regression model in the pytorch environment is carried out.

Tensor splicing and segmentation

torch.cat()
function: splicing tensors according to dimension dim

torch.stack()
function: stitching on the newly created dimension dim

• tensors: sequence of tensors
• dim: the dimension to splice

import torch
a = torch.ones((2,3))
# 张量的拼接
a1 = torch.cat([a,a],dim=0)
print(a)
print(a1)

output:

tensor([[1., 1., 1.],
        [1., 1., 1.]])
tensor([[1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.],
        [1., 1., 1.]])

dim=1

a2 = torch.cat([a,a],dim=1)
print(a2)

tensor([[1., 1., 1., 1., 1., 1.],
        [1., 1., 1., 1., 1., 1.]])

stack：

import torch
a = torch.ones((2,3))

a1 = torch.stack([a,a],dim=2)
print(a1)

output:

tensor([[[1., 1.],
         [1., 1.],
         [1., 1.]],

        [[1., 1.],
         [1., 1.],
         [1., 1.]]])

torch.chunk()
function: averagely split the tensor by dimension dim
Return value: list of tensors
Note: if not divisible, the last tensor is smaller than other tensors

• input: the tensor to split
• chunks: the number of copies to be divided
• dim: the dimension to be split

b = torch.ones((2,5))
print(b)
b1 = torch.chunk(b,dim=1,chunks=2)
print(b1)

output:

tensor([[1., 1., 1., 1., 1.],
        [1., 1., 1., 1., 1.]])
(tensor([[1., 1., 1.],
        [1., 1., 1.]]), tensor([[1., 1.],
        [1., 1.]]))

torch.split()
function: split tensor by dimension dim
return value: tensor list

• tensor: the tensor to split
• split_size_or_setions: When it is an int, it indicates the length of each part; when it is a list, it is split according to the lisr element
• dim: the dimension to be split

c = torch.ones((2,5))
c1 = torch.split(c,2,dim=1)
print(c1)

(tensor([[1., 1.],
        [1., 1.]]), tensor([[1., 1.],
        [1., 1.]]), tensor([[1.],
        [1.]]))

Using list:

c = torch.ones((2,5))
c1 = torch.split(c,[2,1,2],dim=1)
# print(c1)
for i in c1:
    print(i)

tensor([[1., 1.],
        [1., 1.]])
tensor([[1.],
        [1.]])
tensor([[1., 1.],
        [1., 1.]])

Note:
The sum of all elements in the list needs to be equal to the length of the tensor in the specified dimension.

index of the tensor

torch.index_select()
function: on the dimension dim, index data according to index
Return value: tensor spliced ​​according to index index data

• input: Tensor to index
• dim: the dimension to index
• index: the serial number of the data to be indexed

d = torch.randint(0,9,size=(3,3))
idx = torch.tensor([0,2],dtype=torch.long)
d1 = torch.index_select(d,dim=0,index=idx)
print(d)
print(idx)
print(d1)

output:

tensor([[5, 0, 3],
        [1, 0, 3],
        [0, 8, 2]])
tensor([0, 2])
tensor([[5, 0, 3],
        [0, 8, 2]])

Note:
The value of index needs to be a tensor and the dtype must also be torch.long

torch.masked_select()
function: index by True in mask
Return value: one-dimensional tensor

• input: Tensor to index
• mask: Boolean tensor with the same shape as input

t = torch.randint(0,9,size=(3,3))
mask = t.ge(5)
print(t)
print(mask)

The use of the ge(x) method:
when the element in the tensor is >=x, it will become True, and the rest will be False. In this example, x is 5.

The value of t and mask: (will change every time)

tensor([[0, 3, 6],
        [7, 5, 8],
        [2, 8, 0]])
tensor([[False, False,  True],
        [ True,  True,  True],
        [False,  True, False]])

Use the masked_select() method for element indexing where the Boolean value is True

t = torch.randint(0,9,size=(3,3))
mask = t.ge(5)
print(t)
print(mask)

t1 = torch.masked_select(t,mask=mask)
print(t1)

output:

tensor([[8, 8, 4],
        [1, 4, 7],
        [6, 4, 5]])
tensor([[ True,  True, False],
        [False, False,  True],
        [ True, False,  True]])
tensor([8, 8, 7, 6, 5])

tensor transformation

torch.reshape()
function: Transform the shape of the tensor
Note: When the tensor is continuous in memory, the new tensor shares the data memory with the input

• input: Tensor to be transformed
• shape: the shape of the new tensor

t = torch.ones(8)
t_reshape = torch.reshape(t,(2,4))
print(t)
print(t_reshape)

output:

tensor([1., 1., 1., 1., 1., 1., 1., 1.])
tensor([[1., 1., 1., 1.],
        [1., 1., 1., 1.]])

If you change the first element of the original tensor to 2

t[0] = 2
print(t_reshape)

tensor([[2., 1., 1., 1.],
        [1., 1., 1., 1.]])

It will be found that the first element of the tensor whose shape has been changed will also change, indicating that the new tensor shares data memory with input.

Note: Shared data memory does not mean that they are at the same memory address, but that the memory space they point to is the same.

print(id(t),id(t_reshape)) # 2081724628720 2081889008512
print(id(t.data),id(t_reshape.data)) # 1370880600800 1370880600800

torch.transpose()
function: swap the two dimensions of a tensor

torch.t()
function: 2D tensor transpose, for matrix, it is equivalent to torch.transpose(input,0,1)

• input: Tensor to swap
• dim0: the dimension to swap
• dim1: the dimension to swap

t = torch.rand(size=(2,3,4))
print(t)
t1 = torch.transpose(t,0,1)
print(t1)

tensor([[[0.6628, 0.1919, 0.1204, 0.3246],
         [0.0973, 0.0540, 0.3222, 0.4540],
         [0.2753, 0.3575, 0.4117, 0.2105]],

        [[0.0800, 0.7416, 0.3095, 0.6480],
         [0.9503, 0.8973, 0.0828, 0.2698],
         [0.6524, 0.1959, 0.3461, 0.8498]]])
tensor([[[0.6628, 0.1919, 0.1204, 0.3246],
         [0.0800, 0.7416, 0.3095, 0.6480]],

        [[0.0973, 0.0540, 0.3222, 0.4540],
         [0.9503, 0.8973, 0.0828, 0.2698]],

        [[0.2753, 0.3575, 0.4117, 0.2105],
         [0.6524, 0.1959, 0.3461, 0.8498]]])

Process finished with exit code 0

torch.squeeze()
function: Compress dimension (axis) with dimension length 1

• dim: If it is None, remove all axes with a length of 1; if a dimension is specified, it can be removed if and only if the length of the axis is 1;

torch.unsqueeze()
function: expand the dimension according to dim

• dim: the dimension to expand

t = torch.rand((1,2,3,1))
print(t.shape)
t1 = torch.squeeze(t)
print(t1.shape)

output:

torch.Size([1, 2, 3, 1])
torch.Size([2, 3])

If the dim parameter is specified:

t = torch.rand((1,2,3,1))
print(t.shape)
t1 = torch.squeeze(t,dim=0)
print(t1.shape)

output:

torch.Size([1, 2, 3, 1])
torch.Size([2, 3, 1])

It will only compress the axis of the fourth dimension.

Mathematical operations on tensors

Rich tensor mathematical operations are provided in pytorch

1. Addition, subtraction, multiplication and division

• torch.add()
• torch.addcdiv()
• torch.addcmul()
• torch.sub()
• torch.div()
• torch.mul()

2. Logarithmic, exponential, power functions

• torch.log(input,out=None)
• torch.log10(input,out=None)
• torch.log2(input,out=None)
• torch.exp(input,out=None)
• torch.pow()

3. Trigonometric functions

• torch.abs(input,out=None)
• torch.acos(input,out=None)
• torch.cosh(input,out=None)
• torch.cos(input,out=None)
• torch.asin(input,out=None)
• torch.atan(input,out=None)
• torch.atan2(input,out=None)

torch.add()
function: calculate input + alpha x other element by element

• input: the first tensor
• alpha: multiplication factor
• other: the second tensor

The add() method can not only do addition operations, but also linear operations

• Formula in machine learning:
  y = wx + b

import torch
# torch.add()

t = torch.rand((3,3))
t1 = torch.rand((3,3))
print("t",t)
print("t1",t1)
t2 = torch.add(t,t1)
t3 = torch.add(t,t1,alpha=2)
print("t2",t2)
print("t3",t3)

output:

t tensor([[0.8218, 0.8576, 0.4829],
        [0.6046, 0.1290, 0.4835],
        [0.5752, 0.2386, 0.1260]])
t1 tensor([[0.5306, 0.7424, 0.4328],
        [0.3278, 0.1980, 0.4354],
        [0.4742, 0.4319, 0.8922]])
t2 tensor([[1.3524, 1.6000, 0.9157],
        [0.9324, 0.3270, 0.9189],
        [1.0494, 0.6705, 1.0182]])
t3 tensor([[1.8830, 2.3423, 1.3484],
        [1.2602, 0.5249, 1.3544],
        [1.5236, 1.1024, 1.9104]])

linear regression

When linear regressionanalyzeonevariablewith one or morevariablebetweenrelationMethods

Dependent variable: y
Independent variable: x
Relationship: linear

y = wx + b

Analysis: solve for w, b

Solution steps:
1. Determine the model
Model: y = wx + b

2. Select the loss function
MSE: mean square error

3. Solve the gradient and update w, b
w = w - LR w.grad
b = b - LR w.grad

Implementing a one-variable linear regression model in pytorch

# 一元线性回归
import torch
import matplotlib.pyplot as plt

torch.manual_seed(10)

lr = 0.1 #学习率/步长

# 创建训练数据
x = torch.rand(20,1) * 10
y = 2*x + (5 + torch.randn(20,1))

# 构建线性回归参数
w = torch.randn((1),requires_grad=True) # 允许更新梯度
b = torch.zeros((1),requires_grad=True)

for iteration in range(1000):
    # 前向传播
    wx = torch.mul(w,x)
    y_pred = torch.add(wx,b)

    # 计算MSE loss
    loss = (0.5*(y-y_pred)**2).mean()

    # 反向传播
    loss.backward()

    # 更新参数
    w.data.sub_(lr * w.grad)
    b.data.sub_(lr * b.grad)

    if iteration % 20 == 0:
        plt.scatter(x.data.numpy(),y.data.numpy())
        plt.plot(x.data.numpy(),y_pred.data.numpy(),'r-',lw=5)
        plt.text(2,20,'Loss=%.4f' % loss.data.numpy())
        plt.xlim(1.5,10)
        plt.ylim(8,28)
        plt.title("huiguimoxing interation={}".format(iteration))
        plt.pause(0.5)
        plt.show()

        if loss.data.numpy() < 1:
            break

When iterating to 100 times, the loss is close to 0. At this time, the linear graph is also close to these data.