In this blog post, we will explore the concept of linear regression and how to implement it using PyTorch. Regression is a basic statistical modeling technique used to establish the relationship between a dependent variable and one or more independent variables. We will use PyTorch, a popular deep learning framework, to develop and train a linear regression model.
You can find more details about linear concepts here: https://korlakuntasaikamal10.medium.com/understanding-linear-regression-in-machine-learning-bace83acce34
data set
For this analysis, we will use a synthetic dataset generated by the make regression() function in the scikit-learn library. The dataset consists of input features and target variables. The input features represent the independent variables, while the target variable represents the dependent variable we want to predict.
import seaborn as sns
import numpy as sns
import torch
import torch.nn as nn
import torch.optim as optim
import sklearn
from sklearn import datasets
import pandas as pd
# from sklearn we are going to select one dataset
data=datasets.make_regression()
df = pd.DataFrame(data[0], columns=[f"feature_{i+1}" for i in range(data[0].shape[1])])
df["target"] = data[1]PyTorch basics
PyTorch is a powerful open source deep learning framework that provides a flexible way to build and train neural networks. It provides a range of tensor operations, automatic differentiation and optimization algorithms.
PyTorch's dynamic computational graph makes it easy to define and modify complex neural network architectures.
Learn Torch basics here: https://korlakuntasaikamal10.medium.com/pytorch-for-deep-learning-part-1-af4a1daa3454
Prepare data to develop models using sklearn Train-Test-split
x=df.iloc[: , :-1]
y=df.iloc[: , -1]
from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(x,y,test_size=0.2,random_state=42)
print(type(X_train))
# X_train=torch.tensor(X_train,dtype=torch.float32)
X_train = torch.tensor(X_train.values, dtype=torch.float32)
X_test = torch.tensor(X_test.values, dtype=torch.float32)
y_train = torch.tensor(y_train.values, dtype=torch.float32)
y_test = torch.tensor(y_test.values, dtype=torch.float32)Model architecture
Our linear regression model is implemented as a subclass of the nn.Module class in PyTorch. The model consists of multiple fully connected (linear) layers connected in sequence. These layers are responsible for converting input features into predicted output values. The model architecture includes hidden layers of various sizes and an output layer with a single neuron.
# since data is ready we can develop the model:
class linearRegression(nn.Module): # all the dependencies from torch will be given to this class [parent class] # nn.Module contains all the building block of neural networks:
def __init__(self,input_dim):
super(linearRegression,self).__init__() # building connection with parent and child classes
self.fc1=nn.Linear(input_dim,10) # hidden layer 1
self.fc2=nn.Linear(10,5) # hidden layer 2
self.fc3=nn.Linear(5,3) # hidden layer 3
self.fc4=nn.Linear(3,1) # last layer
def forward(self,d):
out=torch.relu(self.fc1(d)) # input * weights + bias for layer 1
out=torch.relu(self.fc2(out)) # input * weights + bias for layer 2
out=torch.relu(self.fc3(out)) # input * weights + bias for layer 3
out=self.fc4(out) # input * weights + bias for last layer
return out # final outcome
input_dim=X_train.shape[1]
torch.manual_seed(42) # to make initilized weights stable:
model=linearRegression(input_dim)training process
To train the model, we use the mean squared error (MSE) loss function, which measures the average squared difference between the predicted value and the actual target value. Optimization is performed using the Adam optimizer, which adjusts the parameters of the model based on the calculated gradients. The model is trained for a specified number of epochs, where each epoch involves forward propagation, loss calculation, backpropagation, and weight updates.
# select loss and optimizers
loss=nn.MSELoss() # loss function
optimizers=optim.Adam(params=model.parameters(),lr=0.01)
# training the model:
num_of_epochs=1000
for i in range(num_of_epochs):
# give the input data to the architecure
y_train_prediction=model(X_train) # model initilizing
loss_value=loss(y_train_prediction.squeeze(),y_train) # find the loss function:
optimizers.zero_grad() # make gradients zero for every iteration so next iteration it will be clear
loss_value.backward() # back propagation
optimizers.step() # update weights in NN
# print the loss in training part:
if i % 10 == 0:
print(f'[epoch:{i}]: The loss value for training part={loss_value}')Evaluation and performance
During training, we monitor the loss values to evaluate the performance of the model. We divided the dataset into training and test sets to evaluate the model's generalization ability. Evaluate the trained model using the test dataset and calculate the test loss. Lower test loss indicates better performance.
# we can do check it with test data:
with torch.no_grad():
model.eval() # make model in evaluation stage
y_test_prediction=model(X_test)
test_loss=loss(y_test_prediction.squeeze(),y_test)
print(f'Test loss value : {test_loss.item():.4f}')# Inference with own data:
pr = torch.tensor(torch.arange(1, 101).unsqueeze(dim=0), dtype=torch.float32).clone().detach()
print(pr)Save and load models
After training, we use PyTorch's save() function to save the parameters of the trained model. This allows us to reuse the model in the future without retraining. We can use the load_state_dict() method to load the saved model and make predictions on new data.
# save the torch model:
from pathlib import Path
filename=Path('models')
filename.mkdir(parents=True,exist_ok=True)
model_name='linear_regression.pth' # model name
# saving path
saving_path=filename/model_name
print(saving_path)
torch.save(obj=model.state_dict(),f=saving_path)
# we can load the saved model and do the inference again:
load_model=linearRegression(input_dim) # creating an instance again for loaded model
load_model.load_state_dict(torch.load('/content/models/linear_regression.pth'))
load_model.eval() # make model in evaluation stage
with torch.no_grad():
pred = load_model(torch.tensor([[ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10., 11., 12.,
13., 14., 15., 16., 17., 18., 19., 20., 21., 22., 23., 24.,
25., 26., 27., 28., 29., 30., 31., 32., 33., 34., 35., 36.,
37., 38., 39., 40., 41., 42., 43., 44., 45., 46., 47., 48.,
49., 50., 51., 52., 53., 54., 55., 56., 57., 58., 59., 60.,
61., 62., 63., 64., 65., 66., 67., 68., 69., 70., 71., 72.,
73., 74., 75., 76., 77., 78., 79., 80., 81., 82., 83., 84.,
85., 86., 87., 88., 89., 90., 91., 92., 93., 94., 95., 96.,
97., 98., 99., 100.]]))
print(f'prediction value : {pred.item()}')in conclusion
In this blog post, we explore linear regression analysis using PyTorch. We learned how to prepare a dataset, build a linear regression model, train the model using gradient descent optimization, and evaluate its performance. PyTorch's flexibility and broad functionality make it a powerful tool for developing and training a variety of machine learning models.
☆ END ☆
If you see this, it means you like this article, please forward it and like it. Search "uncle_pn" on WeChat. Welcome to add the editor's WeChat "woshicver". A high-quality blog post will be updated in the circle of friends every day.
↓Scan the QR code to add the editor↓
