Python and Java code implementation: tangent method to solve one-dimensional optimization problems

Principle of tangent method

In the previous article , we used the golden section method to solve a one-dimensional optimization problem.
This article introduces another algorithm for solving this problem: the tangent method.
The logic of this method is: first in $[a,b]$ Randomly select a point$a_k$, make a tangent along the point, and the tangent and $The\; intersection\; of\; the\; x-$ axis is chosen as the next point$a_{k+1}$, repeat until $|{\alpha }_{k+1}-a_k|$ is less than a given threshold, the optimal solution is obtained.

Next, let's look at the mathematics of the method.
For point $a_k$, the tangent equation is
$$y=f^{\prime }(a_k)+f^{\prime \prime }\left(a_k\right)(a-a_k)$$
令$y=0$ , get the corresponding$a_{k+1}$为
$$a_{k+1}=a_k-\frac{f^{\prime }(a_k)}{f^{\prime \prime }\left(a_k\right)}$$

with $The\; increase\; of\; k$ ,$f^{\prime }\left(a_k\right)$ can tend to 0,$|{\alpha }_{k+1}-a_k|$ can be less than any given threshold, and finally get the optimal solution.
Of course, if the first or second order of the function to be optimized is not differentiable, then the tangent method cannot be used.

Code

Python code

The following tangent_method function is the Python code of the tangent method.
In addition to the function to be optimized $In\; addition\; to\; f\left(t\right)$ , the code also defines the corresponding first derivative$d\_f\left(t\right)$ and the second derivative$d\_2\_f(t)$。

# 待优化函数
def f(t):
    return t ** 2 - t * 5 + 8


# 待优化函数一阶导数
def d_f(t):
    return 2 * t - 5


# 待优化函数二阶导数
def d_2_f(t):
    return 2


# 切线法
def tangent_method(x, eps):

    cnt = 0
    while abs(-d_f(x) / d_2_f(x)) > eps:
        # 更新x，并增加迭代次数
        x += -d_f(x) / d_2_f(x)
        cnt += 1

    return x, f(x), cnt


if __name__ == '__main__':
    # 参数设置
    left_point = 1
    right_point = 7
    min_interval_value = 0.1

    # 选取初始点
    x0 = 6
    # 调用切线法求解最小值
    best_x, best_y, iter_cnt = tangent_method(x0, min_interval_value)
    # 输出最优解
    print('best_x: {}, best_y: {}, iter_cnt: {}.'.format(best_x, best_y, iter_cnt))

java code

The following tangentMethod function is the Java code of the tangent method.

public class TangentMethod {
    
    

    public static void main(String[] args) {
    
    
        // 参数设置
        int leftPoint = 1;
        int rightPoint = 7;
        double minIntervalValue = 0.1;

        double x0 = 6.0;
        Solution best_solution = tangentMethod(x0, minIntervalValue);
        System.out.println("best_x: " + best_solution.best_x);
        System.out.println("best_y: " + best_solution.best_y);
        System.out.println("cnt: " + best_solution.cnt);
    }

    // 切线法
    private static Solution tangentMethod(double x, double eps) {
    
    
        // 统计迭代次数
        int cnt = 0;

        while (Math.abs(df(x) / d2f(x)) > eps) {
    
    
            // 更新x，并增加迭代次数
            x -= df(x) / d2f(x);
            cnt ++;
        }

        // 构造最优解对象
        Solution best_solution = new Solution();
        best_solution.best_x = x;
        best_solution.best_y = f(x);
        best_solution.cnt = cnt;

        return best_solution;
    }

    // 待优化函数
    private static double f(double t) {
    
    
        return t * t - t * 5 + 8;
    }

    // 待优化函数一阶导数
    private static double df(double t) {
    
    
        return t * 2 - 5;
    }

    // 待优化函数二阶导数
    private static double d2f(double t) {
    
    
        return 2;
    }

    // 解对象
    private static class Solution {
    
    
        double best_x;
        double best_y;
        int cnt;
    }
}

Solution example

Regardless of running a Python program or a Java program, the optimal solution can be obtained as follows:

best_x: 2.5, best_y: 1.75, iter_cnt: 1.

It can be seen from the results that the tangent method only needs one iteration to obtain the optimal solution;
while the golden section method requires nine iterations.
This is mainly because the tangent method uses more information about the function to be optimized, including the first and second derivatives, so the calculation efficiency is higher, which is also mentioned in the article .

In addition, compared with the golden section method, the tangent method does not depend on the values ​​of the left and right endpoints, but it is more sensitive to the initial value (although the examples in this article are not sensitive to the initial value), which is also worth noting.