Gradient descent is a very commonly used algorithm. It is not only used in linear regression and linear regression models, but also in squared error cost functions.
In this section, we will combine gradient descent with cost functions. We will use this algorithm and apply it to a specific linear regression algorithm for fitting a straight line.
The comparison between gradient descent algorithm and linear regression algorithm is as follows:
Using the gradient descent method on our previous linear regression problem, the key is to find the derivative of the cost function, that is:
The algorithm is rewritten as:
The algorithm we just used is sometimes called batch gradient descent. In fact, in machine learning, algorithms are usually not given names, but the name "batch gradient descent" refers to the fact that in each step of gradient descent, we use all training samples. In gradient descent , when calculating the differential derivative term, we need to perform a summation operation, so in each individual gradient descent, we eventually have to calculate such a thing, which requires the sum of all training samples. Therefore, the name batch gradient descent method indicates that we need to consider all this "batch" training samples. In fact, sometimes there are other types of gradient descent methods that are not this "batch" type and do not consider the entire training set. , but only focus on some small subsets of the training set each time. We will also introduce these methods in later lessons.
But for now, applying the algorithm you just learnedÿ