The knowledge points about logistic regression have been mentioned before, and those who want to consolidate and learn more can read this article:
[Machine Learning] Logistic Regression - Wugui.lsy's Blog - CSDN Blog
language: python
Framework: pytorch
Tool: Anaconda 3
Let's use an example to learn Logistic regression in detail:
import torch
from torch.autograd import Variable
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline#设置随机种子
torch.manual_seed(2017)#从 data.txt 中读入点
with open('./data.txt','r') as f:
data_list=[i.split('\n')[0].split(',')for i in f.readlines()]
data = [(float(i[0]),float(i[1]),float(i[2])) for i in data_list]
#标准化
x0_max=max([i[0]for i in data])
x1_max=max([i[1] for i in data])
data =[(i[0]/x0_max,i[1]/x1_max,i[2] ) for i in data]
x0=list(filter(lambda x:x[-1] ==0.0,data)) #选择第一类的点
x1=list(filter(lambda x:x[-1]==1.0,data)) #选择第二类的点
plot_x0 = [i[0] for i in x0]
plot_y0 = [i[1]for i in x0]
plot_x1 = [i[0] for i in x1]
plot_y1 = [i[1] for i in x1]
plt.plot(plot_x0,plot_y0,'ro',label='x_0')
plt.plot(plot_x1,plot_y1,'bo',label='x_1')
plt.legend(loc='best'),L 
Next, we convert the data into Numpy type, and then convert to tensor to prepare for subsequent training
np_data = np.array(data,dtype='float32') #转换成numpy array
x_data = torch.from_numpy(np_data[:,0:2]) #转换成tensor,大小是【100，2】
y_data = torch.from_numpy(np_data[:,-1]).unsqueeze(1) #转换成tensor,大小为【100，1】Let's define the sigmoid function:
def sigmoid(x):
return 1/(1+np.exp(-x))Draw the sigmoid function:
#画出sigmoid函数
plot_x=np.arange(-10,10.01,0.01)
plot_y=sigmoid(plot_x)
plt.plot(plot_x,plot_y,'r')
x_data = Variable(x_data)
y_data = Variable(y_data)In Pytorch, we don’t need to write sigmoid functions ourselves. Pytorch has already written some commonly used functions for us in the underlying C++ language, which is not only convenient for us to use, but also faster and more stable than our own implementation. . Use by importing torch.nn.functional, the following is how to use it
import torch.nn.functional as F# 定义 logistic 回归模型
w = Variable(torch.randn(2, 1), requires_grad=True)
b = Variable(torch.zeros(1), requires_grad=True)
def logistic_regression(x):
return torch.sigmoid(torch.mm(x, w) + b)Before the update, we can plot the effect of the classification:
# 计算loss
def binary_loss(y_pred, y):
logits = (y * y_pred.clamp(1e-12).log() + (1 - y) * (1 - y_pred).clamp(1e-12).log()).mean()
return -logitsy_pred = logistic_regression(x_data)
loss = binary_loss(y_pred, y_data)
print(loss)tensor(0.6363, grad_fn=<NegBackward0>)
After getting the loss, we can use the gradient descent method to update the parameters
# 自动求导并更新参数
loss.backward()
w.data = w.data - 0.1 * w.grad.data
b.data = b.data - 0.1 * b.grad.data
# 算出一次更新之后的loss
y_pred = logistic_regression(x_data)
loss = binary_loss(y_pred, y_data)
print(loss)tensor(0.6360, grad_fn=<NegBackward0>)
The above parameter update method is actually a tedious and repetitive operation. If we have many parameters, such as 100, then we need to write 100 lines to update the parameters. For convenience, we can write a function to update. In fact, PyTorch has already encapsulated it for us. There is a function to do this, which is the optimizer in torch.optimPyTorch¶
The use torch.optim requires another data type, that is nn.Parameter, this is essentially the same as Variable, but nn.Parameter the default is to ask for gradients, while Variable does not ask for gradients by default.
The gradient descent method can be used torch.optim.SGD to update the parameters. The optimizer in PyTorch has more optimization algorithms, which we will introduce in more detail later.
After putting the parameters w and b torch.optim.SGD in, explain the size of the learning rate, and then you can use it optimizer.step() to update the parameters. For example, we will pass the parameters into the optimizer below, and set the learning rate to 1.0
# 使用 torch.optim 更新参数
from torch import nn
w = nn.Parameter(torch.randn(2, 1))
b = nn.Parameter(torch.zeros(1))
def logistic_regression(x):
return torch.sigmoid(torch.mm(x, w) + b)
optimizer = torch.optim.SGD([w, b], lr=1.)# 进行 1000 次更新
import time
start = time.time()
for e in range(1000):
# 前向传播
y_pred = logistic_regression(x_data)
loss = binary_loss(y_pred, y_data) # 计算 loss
# 反向传播
optimizer.zero_grad() # 使用优化器将梯度归 0
loss.backward()
optimizer.step() # 使用优化器来更新参数
# 计算正确率
mask = y_pred.ge(0.5).float()
acc = (mask == y_data).sum().item() / y_data.shape[0]
if (e + 1) % 200 == 0:
print('epoch: {}, Loss: {:.5f}, Acc: {:.5f}'.format(e+1, loss.item(), acc))
during = time.time() - start
print()
print('During Time: {:.3f} s'.format(during))
epoch: 200, Loss: 0.39401, Acc: 0.91000 epoch: 400, Loss: 0.32355, Acc: 0.91000 epoch: 600, Loss: 0.29010, Acc: 0.91000 epoch: 800, Loss: 0.27043, Acc: 0.91000 epoch: 1000, Loss: 0.25741, Acc: 0.90000 During Time: 0.354 s
It can be seen that it is very simple to update the parameters after using the optimizer. You only need to use the return to zero gradient before the automatic derivation optimizer.zero_grad() , and then use it optimizer.step()to update the parameters. It is very simple and after 1000 updates, the loss has also dropped to a relatively low .
Below we draw the updated results
It can be seen that after the update, the model has been able to basically separate these two types of points¶
We used the loss we wrote earlier. In fact, PyTorch has written some common losses for us. For example, the loss in linear regression is , nn.MSE()and the binary classification loss in Logistic regression is in PyTorch nn.BCEWithLogitsLoss(). For more losses, you can check document
The loss function that PyTorch implements for us has two advantages. The first is that it is convenient for us to use without reinventing the wheel. The second is that its implementation is in the underlying C++ language, so it is faster and more stable than our own. achieve better
In addition, for the sake of stability, PyTorch combines the Sigmoid operation and the final loss of the model nn.BCEWithLogitsLoss(), so we don’t need to add the Sigmoid operation when we use the loss that comes with PyTorch
# 使用自带的loss
criterion = nn.BCEWithLogitsLoss() # 将 sigmoid 和 loss 写在一层，有更快的速度、更好的稳定性
w = nn.Parameter(torch.randn(2, 1))
b = nn.Parameter(torch.zeros(1))
def logistic_reg(x):
return torch.mm(x, w) + b
optimizer = torch.optim.SGD([w, b], 1.)y_pred = logistic_reg(x_data)
loss = criterion(y_pred, y_data)
print(loss.data)tensor(0.7911)
# 同样进行 1000 次更新
start = time.time()
for e in range(1000):
# 前向传播
y_pred = logistic_reg(x_data)
loss = criterion(y_pred, y_data)
# 反向传播
optimizer.zero_grad()
loss.backward()
optimizer.step()
# 计算正确率
mask = y_pred.ge(0.5).float()
acc = (mask == y_data).sum().item() / y_data.shape[0]
if (e + 1) % 200 == 0:
print('epoch: {}, Loss: {:.5f}, Acc: {:.5f}'.format(e+1, loss.item(), acc))
during = time.time() - start
print()
print('During Time: {:.3f} s'.format(during))epoch: 200, Loss: 0.32778, Acc: 0.88000 epoch: 400, Loss: 0.29239, Acc: 0.87000 epoch: 600, Loss: 0.27187, Acc: 0.87000 epoch: 800, Loss: 0.25840, Acc: 0.87000 epoch: 1000, Loss: 0.24887, Acc: 0.89000 During Time: 0.224 s