Table of contents
5. Drawing the decision boundary
1. Import library
Machine learning tasks fall into two broad categories: classification and regression
Classification is to identify and classify a bunch of targets, such as cat and dog classification, handwritten digit classification, etc.
Regression is the prediction of the next behavior of something, such as predicting the weather, etc.
The task we are going to complete this time is logistic regression. Although the name is regression, it is actually a binary classification task.
First look at the library files we need
torch.nn is an interface specifically designed for neural networks
matplotlib is used to draw images and help with visualization tasks
torch defines tensors, and data transmission is realized by using tensors
optim package of optimizers, such as SGD, etc.
A package for numpy data processing
2. Define the dataset
Briefly explain the task, I want to generate several points in a square area, then manually design the label, and finally draw the decision boundary through the training of the neural network
Assumption: the side length of the square is 2, the coordinates of the lower left corner are (0, 0), and the coordinates of the upper right corner are (2, 2)
Then we manually define the dividing line y = x, which is defined as blue above the dividing line and red below it
2.1 Generate data
The first code to generate data is
First generate 200 points through rand (uniform distribution of 0-1), and expand them by 2 times, x1 represents the abscissa, x2 represents the ordinate
Then define the classification, here is a brief introduction to the zip function.
zip will pack a and b here into a pair, so that i corresponds to (1, 'a'), and i[0] corresponds to 1 2 3
Going back to our code, because what we want to achieve is binary classification, so we define two different types and store them with pos and neg. Then we know that i[1] represents x2, and i[0] represents x1, so x2 - x1 < 0 means that it is pos type under the straight line y=x. Otherwise, it is of type neg
Finally, we need to draw the pos and neg types. Because the coordinates in pos are actually similar to (1,1), because the i in pos.append(i) is actually the coordinate form of (x1,x2), so we define the first element x1 in pos as The value is assigned to the abscissa, and the second element x2 is assigned to the ordinate
Then use scatter to draw discrete points, draw pos in red color, and neg in blue color, as shown in the figure
2.2 Set the label
What we are doing is actually supervised learning, so we need labels
It should be noted here that, unlike regression tasks, x1 is not an input and x2 is not an output. Both x1 and x2 should be input elements, that is, feature features . So we should set a label for the red point set, such as 1, and set a label for the blue point set, such as 0.
The implementation code is as follows
It is easy to understand that the training set x_data should be all samples, that is, all elements of pos and neg. As previously introduced, x1 and x2 are both input features, so the shape of x_data should be [200,2]. And y_data only has 1 (pos red) category, or 0 (neg blue) type, so the shape of y_data should be [200,1]. The reason for the y_data view is to be in the form of a matrix instead of a vector
What this means is that if the coordinates are (1.5,0.5) then it should fall in the red area, then the label of this point is 1
3. Build network + optimizer
The type of network is very simple and will not be repeated here. As for why you want to inherit nn.Module or super, you don’t need to worry about it. Basically, it’s written like this, just remember it.
It should be noted that the feature we input is (n * 2), so Linear should be (2,1)
The final output of binary classification generally uses the sigmoid function
The loss function here we choose BCE, binary cross entropy loss function.
The algorithm is stochastic gradient descent
4. Training
The training process is also relatively simple, which is to compare the predicted output value of the model with the real label. The gradient is then zeroed, backpropagated and the gradient updated.
5. Drawing the decision boundary
After the model training is completed here, take out w0, w1, b, and then draw a straight line
What is to be drawn here is w0 * x1+ w1 * x2 + b = 0, because it was first introduced that x1 represents the abscissa x, and x2 represents the ordinate y. Through deformation, it can be known that y = (- w0 * x1 - b ) / w1, the result is shown in the figure
The loss of the program output is
Finally, w0 = 4.1911, w1 = -4.0290, b = 0.0209, approximately equal to y = x, similar to the dividing line we just defined
6. Code
import torch.nn as nn
import matplotlib.pyplot as plt
import torch
from torch import optim
import numpy as np
torch.manual_seed(1) # 保证程序随机生成数一样
x1 = torch.rand(200) * 2
x2 = torch.rand(200) * 2
data = zip(x1,x2)
pos = [] # 定义类型 1
neg = [] # 定义类型 2
def classification(data):
for i in data:
if(i[1] - i[0] < 0):
pos.append(i)
else:
neg.append(i)
classification(data)
pos_x = [i[0] for i in pos]
pos_y = [i[1] for i in pos]
neg_x = [i[0] for i in neg]
neg_y = [i[1] for i in neg]
plt.scatter(pos_x,pos_y,c='r')
plt.scatter(neg_x,neg_y,c='b')
plt.show()
x_data = [[i[0],i[1]] for i in pos]
x_data.extend([[i[0],i[1]] for i in neg])
x_data = torch.Tensor(x_data) # 输入数据 feature
y_data = [1 for i in range(len(pos))]
y_data.extend([0 for i in range(len(neg))])
y_data = torch.Tensor(y_data).view(-1,1) # 对应的标签
class LogisticRegressionModel(nn.Module): # 定义网络
def __init__(self):
super(LogisticRegressionModel,self).__init__()
self.linear = nn.Linear(2,1)
self.sigmoid = nn.Sigmoid()
def forward(self,x):
x = self.linear(x)
x = self.sigmoid(x)
return x
model = LogisticRegressionModel()
criterion = nn.BCELoss()
optimizer = optim.SGD(model.parameters(),lr =0.01)
for epoch in range(10000):
y_pred = model(x_data)
loss = criterion(y_pred,y_data) # 计算损失值
if epoch % 1000 == 0:
print(epoch,loss.item()) # 打印损失值
optimizer.zero_grad() # 梯度清零
loss.backward() # 反向传播
optimizer.step() # 梯度更新
w = model.linear.weight[0] # 取出训练完成的结果
w0 = w[0]
w1 = w[1]
b = model.linear.bias.item()
with torch.no_grad(): # 绘制决策边界，这里不需要计算梯度
x= torch.arange(0,3).view(-1,1)
y = (- w0 * x - b) / w1
plt.plot(x.numpy(),y.numpy())
plt.scatter(pos_x,pos_y,c='r')
plt.scatter(neg_x,neg_y,c='b')
plt.xlim(0,2)
plt.ylim(0,2)
plt.show()
program results
