Detailed explanation of the gradient descent method of Wu Enda's machine learning notes

Gradient descent is an algorithm used to find the minimum value of a function. We will use the gradient descent algorithm to find the minimum value of the cost function insert image description here.

The idea behind gradient descent is that we start by randomly selecting a insert image description herecombination of parameters, calculate the cost function, and then we look for the next combination of parameters that reduces the value of the cost function the most. We continue to do this until we reach a local minimum (local minimum). Because we have not tried all parameter combinations, we cannot be sure whether the local minimum we obtained is the global minimum (global minimum). Choose a different Depending on the initial parameter combination, different local minima may be found.

insert image description hereImagine you are standing at this point on the mountain, standing on this red hill in your imaginary park. In the gradient descent algorithm, what we have to do is rotate 360 ​​degrees, look around us and ask ourselves where we want to be. In one direction, use small steps to go down the mountain as soon as possible. In what direction do these little steps need to go? If we stand at this point on the hillside, and you look around, you will find the best direction to go down the mountain. You look around again, and then think about it again, from which direction should I take small steps down the mountain? Then you take another step according to your own judgment and repeat the above steps. From this new point, you look around and decide in which direction you will go down the mountain fastest, then you take another small step, and so on. until you are close to the location of the local minimum.

Guess you like

Origin blog.csdn.net/zy_dreamer/article/details/132737898