Table of contents
1. Configure the virtual environment
2. Library version introduction
1. Define linear model linear_model
2. Define the loss function loss_function
1. Experiment introduction
- Implemented using Pytorch
- Linear model building
- Construct loss function
- Calculate loss value
2. Experimental environment
This series of experiments uses the PyTorch deep learning framework. The relevant operations are as follows:
1. Configure the virtual environment
conda create -n DL python=3.7 conda activate DLpip install torch==1.8.1+cu102 torchvision==0.9.1+cu102 torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html
conda install matplotlib conda install scikit-learn2. Library version introduction
| software package | This experimental version | The latest version currently |
| matplotlib | 3.5.3 | 3.8.0 |
| numpy | 1.21.6 | 1.26.0 |
| python | 3.7.16 | |
| scikit-learn | 0.22.1 | 1.3.0 |
| torch | 1.8.1+cu102 | 2.0.1 |
| torchaudio | 0.8.1 | 2.0.2 |
| torchvision | 0.9.1+cu102 | 0.15.2 |
3. Experimental content
ChatGPT：
The linear model is a basic machine learning model that is used to establish a linear relationship between input features and output. It is a linear combination model that predicts the output value by weighting the sum of input features and adding a bias term.
The general form of the linear model can be expressed as: y = w1x1 + w2x2 + ... + wnxn + b, where y is the output variable, x1, x2, ..., xn are the input features, w1, w2, ..., wn is the weight of the feature, and b is the bias term. The goal of the model is to minimize the difference between the predicted value and the true value by adjusting the weights and bias terms.
There are several common application forms of linear models:
Linear Regression: used to establish a linear relationship between input features and continuous output. It fits the best regression line by minimizing the squared difference between the predicted value and the true value.
Logistic Regression: used to establish a linear relationship between input features and binary or multi-class output. It makes classification predictions by mapping the results of linear combinations to probability values using logistic functions (such as sigmoid functions).
Support Vector Machines (SVM): used for binary classification and multi-classification problems. SVM separates samples of different categories by finding an optimal hyperplane. It can use different kernel functions to handle nonlinear problems.
Ridge Regression and Lasso Regression: used to deal with regression problems with multicollinearity. By introducing regularization terms to the weights, they can reduce the influence of features and improve the generalization ability of the model.
The advantages of linear models include simplicity, ease of interpretation, and computational efficiency. They have wide applications in many practical problems. However, linear models also have some limitations, such as their weak ability to model nonlinear relationships. When dealing with complex problems, the performance of linear models can be improved by introducing nonlinear feature transformations or extending them with kernel functions.
This series is experimental content and does not explain theoretical knowledge in detail.
(Ahem, I actually don’t have time to sort it out. I’ll come back and fill in the gaps when I have the opportunity)
0. Import library
import torch1. Define a linear modellinear_model
This function accepts input data x, uses randomly generated weights wand biases b, and calculates an output value output. The form of the linear model here is output = x * w + b.
def linear_model(x):
w = torch.rand(1, 1, requires_grad=True)
b = torch.randn(1, requires_grad=True)
return torch.matmul(x, w) + b2. Define the loss functionloss_function
The mean square error (MSE) is used here as the loss function to calculate the square of the difference between the predicted value and the true value.
def loss_function(y_true, y_pred):
loss = (y_pred - y_true) ** 2
return loss3. Define data
-
Generate a random input tensor
xwith a shape of (5, 1), indicating that there are 5 samples and the feature dimension of each sample is 1. -
Generate a target tensor
ywith shape (5, 1), representing the corresponding true label. - Prints information about the data, including input
xand target values for each sampley.
x = torch.rand(5, 1)
y = torch.tensor([1, -1, 1, -1, 1], dtype=torch.float32).view(-1, 1)
print("The data is as follows:")
for i in range(x.shape[0]):
print("Item " + str(i), "x:", x[i][0], "y:", y[i])
4. Call the model
-
Use the function to predict
linear_modelthe input and get the prediction result .xprediction -
The loss tensor is obtained by
loss_functioncalculating the loss between the predicted result and the true labelloss. - The loss value for each sample is printed.
prediction = linear_model(x)
loss = loss_function(y, prediction)
print("The all loss value is:")
for i in range(len(loss)):
print("Item ", str(i), "Loss:", loss[i])
5. Complete code
import torch
def linear_model(x):
w = torch.rand(1, 1, requires_grad=True)
b = torch.randn(1, requires_grad=True)
return torch.matmul(x, w) + b
def loss_function(y_true, y_pred):
loss = (y_pred - y_true) ** 2
return loss
x = torch.rand(5, 1)
y = torch.tensor([1, -1, 1, -1, 1], dtype=torch.float32).view(-1, 1)
print("The data is as follows:")
for i in range(x.shape[0]):
print("Item " + str(i), "x:", x[i][0], "y:", y[i])
prediction = linear_model(x)
loss = loss_function(y, prediction)
print("The all loss value is:")
for i in range(len(loss)):
print("Item ", str(i), "Loss:", loss[i])
Notice:
The linear model in this experiment simply used random weights and biases to calculate the mean square error loss of the model on the training set, and did not use an optimization algorithm to update the model parameters.
Usually, optimization algorithms such as gradient descent are used to minimize the loss function, and the parameters of the model are continuously updated based on the training data. Please listen to the next chapter for details.