Table of contents
1. Line search algorithm for unconstrained differentiable optimization problems
2. Monotone line search criterion
3. Non-monotone line search criteria
2. Convergence of line search algorithm
Part of the content of this article comes from the textbook
1. Line search algorithm for unconstrained differentiable optimization problems
Line search algorithm:
This type of algorithm can be understood as consisting of two parts: step size α and direction d.
The step size is also called the learning rate.
An inappropriate step size will either result in failure to converge or converge to a non-optimal solution.
The line search criterion is to study the conditions under which the step size can finally converge to the optimal solution.
Line search algorithms can be divided into monotonic line search algorithms and non-monotone line search algorithms. Monotone refers to whether each step is better and monotonic than the previous step.
The corresponding criteria are also divided into monotonic line search criteria and non-monotone line search criteria.
2. Monotone line search criterion
1. Armijo Guidelines
α=0 obviously satisfies the Armijo criterion, so the Armijo criterion is not a sufficient condition.
Armijo criteria are generally not used alone, but as part of other criteria.
2. Goldstein’s criterion
3. Wolfe’s criterion
3. Non-monotone line search criteria
1.Grippo guidelines
2. ZhangHager criterion
4. Convergence analysis
1，Zoutendijk定理
2. Convergence of line search algorithm
