[Python] machine learning univariate linear regression using the batch gradient descent to find the appropriate parameter values

[Python] machine learning univariate linear regression using the batch gradient descent to find the appropriate parameter values

---

 

This title from Andrew Ng machine learning video.

 

topic:

Are you a restaurant owner, you want to open branches in other cities, so you get some data (data below the article most), the data included in profit caused by a different city population and the city. The first column is the number of the city's population, the second column is the number of Profit in the city shop brings.

Now, assuming a start θ0 and θ1 are 0, using a gradient descent method, to find a suitable value of θ, where the learning rate α = 0.01, iterative rounds of 1000

 

Last article, we have come to CostFunction, namely the loss of function.

 

Now we need to find to make minimum loss function value of θ, gradient descent function

 

1, the leader packet

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

 

2, written before CostFunction function

def computeCost(X, y, theta):
    inner = np.power(((X * theta.T) - y), 2)
    return np.sum(inner) / (2 * len(X))

 

3, the introduction of documents to separate X and Y, plus a 1 in the left X, and theta] 0 is set to θ1 0,0

path = 'ex1data1.txt'
data = pd.read_csv(path, header=None, names=['Population', 'Profit'])

data.insert(0, 'Ones', 1)
rows = data.shape[0]
cols = data.shape[1]
X = data.iloc[:, 0:cols - 1]
Y = data.iloc[:, cols - 1:cols]
theta = np.mat('0,0')
X = np.mat(X.values)
Y = np.mat(Y.values)

cost = computeCost(X, Y, theta)

 

4, set update rate α of 0.01, the number of iterations set to 1000

alpha = 0.01 
iters = 1500

 

5, write to achieve a gradient descent function

DEF gradientDescent (X-, the Y, Theta, Alpha, iters): 
    TEMP = np.mat (np.zeros (theta.shape))   # an array, temp θ size is the number of 
    parameters = int (theta.ravel (). Shape [. 1])   # number of parameters 
    cost = np.zeros (iters)   # an array, with each memory costFunction calculated values 

    for I in Range (iters): 
        error = (X-theta.T *) - the Y;   # error value 
        for J in Range (Parameters): 
            Term = np.multiply (error, X-[:, J]) 
            TEMP [0, J] = Theta [0, J] - ((Alpha / len (X-) ) * np.sum (Term)) 
        Theta = temp
        cost[i] = computeCost(X,Y,theta)
    return theta, cost

Resolution:

temp is a temporary variable stored array, because all the required synchronization update θ, so the first stored in the temporary variable, θ rear complete calculation of all the values ​​until synchronization is updated.

int value is a number of parameters, i.e. the number of variables, and this problem has θ0 theta] 1, so that parameters = 2

is an array of cost, size, and the same number of iterations, each layer stored in the current iteration CostFunction return value

 

6, call the function, and returns the result

g, cost = gradientDescent(X, Y, theta, alpha, iters)
print(g)

Final results g = [[- 3.24140214 1.1272942]]

I.e., the last θ0 = -3.24 θ1 = 1.127

 

7, break out the map, to see if convergence

fig, ax = plt.subplots(figsize=(12,8))
ax.plot(np.arange(iters),cost,'r')
ax.set_xlabel('Iterations')
ax.set_ylabel('Cost')
plt.show()

 

 

With the enlargement of the number of iterations iters, the loss slowly decreased, so effective, calculated correctly.