Python and Java code implementation: advance and retreat method to determine the search interval

introduction

As introduced in the previous article , the golden section method can solve the following one-dimensional optimization problem: within a search interval, the minimum value point of the objective function is included. If there is only one initial point of the function $x_0$, but do not know where the minimum value is in the search interval, then you can first use the advance and retreat method that will be introduced in this article to determine the search interval, and then use the golden section method to obtain the extreme value.

The principle of advance and retreat

The theoretical basis of the advance and retreat method is: assuming $f\left(x\right)$ is a unimodal function with only one minimum value, and the interval$[a,b]$ is a search interval of the minimum value, then for any$x_1,x_2\in [a,b]$，如果$f(x_1)<f\left(x_2\right)$ , then the search interval can be shrunk to$[a,x_2]$ ; if$f(x_1)>f\left(x_2\right)$ , then the search interval can be shrunk to$[x_1,b]$。

Therefore, if given an initial point $x_0$and the search step size $In\; the\; case\; of\; h$ , first search one step forward with the initial step size, and calculate the corresponding value$f(x_0+h)$:

（1）如果$f\left(x_0\right)<f(x_0+h)$ , it can be deduced that the search interval is$[\tilde{x},x_0+h]$ , where$\tilde{x}$ Although it is temporarily unknown, the right endpoint can be determined as$x_0+h$ ; take a step back and calculate$f(x_0-\lambda h)$ until a$\lambda$， let$f(x_0-\lambda h)>f\left(x_0\right)$ , so as to determine the search interval as$[f(x_0-\lambda h),f(x_0+h)]$。

(2) If $f(x_0)>f(x_0+h)$ , it can be deduced that the search interval is$[x_0,\tilde{x}]$, where$\tilde{x}$ To be requested, further calculate$f(x_0+\lambda h)$ until a$\lambda$，使得$f(x_0+\lambda h)>f(x_0+h)$ , so as to determine the search interval as$[f(x_0),f(x_0+\lambda h)]$。

(3) If $f(x_0)=f(x_0+h)$ , since$f\left(x\right)$ is a unimodal function, so the search interval can be directly determined as$[f(x_0),f(x_0+h)]$。

Python code implementation

The one-dimensional optimization problem still uses the example in the golden section method:
$$min(t^2-5t+8)$$
The minimum value of this problem is 1.75, and its corresponding$t=2.5$。

The Python code to implement the above advance and retreat method is as follows.

# 待优化函数
def f(t):
    return t ** 2 - t * 5 + 8


# 进退法：确定搜索区间
def advance_and_retreat(func, x, h):

    if abs(func(x) - func(x + h)) <= 1e-6:
        # 第三种情况
        x_min, x_max = x, x + h
    elif func(x) < func(x + h):
        # 第一种情况
        x_max = x + h
        lamb = 1
        while func(x - lamb * h) < func(x):
            lamb += 1
        x_min = x - lamb * h
    else:
        # 第二组情况
        x_min = x + h
        lamb = 2
        while func(x + lamb * h) < func(x + h):
            lamb += 1
        x_max = x + lamb * h

    return x_min, x_max


if __name__ == '__main__':

    # 步长
    step = 0.1

    # 第一种情况实例
    x1 = 2
    lb1, ub1 = advance_and_retreat(f, x1, step)
    print('S1: x0 = {}, lb = {}, ub = {}'.format(x1, lb1, ub1))

    # 第二种情况实例
    x2 = 4
    lb2, ub2 = advance_and_retreat(f, x2, step)
    print('S2: x0 = {}, lb = {}, ub = {}'.format(x2, lb2, ub2))

    # 第三种情况实例
    x3 = 2.45
    lb3, ub3 = advance_and_retreat(f, x3, step)
    print('S3: x0 = {}, lb = {}, ub = {}'.format(x3, lb3, ub3))

In the main function, three different initial values ​​are selected, corresponding to the three situations described in the principle. After running the above code, you can get the following results. Obviously, each interval contains the extreme point 2.5, that is, the correct search interval is obtained.

S1: x0 = 2, lb = 2.1, ub = 2.9
S2: x0 = 4, lb = 1.0, ub = 4.1
S3: x0 = 2.45, lb = 2.45, ub = 2.5500000000000003

Java code implementation

The Java code is shown below.

public class AdvanceAndRetreat {
    
    
    public static void main(String[] args) {
    
    
        // 步长
        double step = 0.1;

        // 第一种情况实例
        double x1 = 2.0;
        Intervals interval1 = advanceAndRetreatMethod(x1, step);
        System.out.println("S1: x0 = " + x1 + ", lb = " + interval1.lb +  ", ub = " + interval1.ub);

        // 第二种情况实例
        double x2 = 4;
        Intervals interval2 = advanceAndRetreatMethod(x2, step);
        System.out.println("S2: x0 = " + x2 + ", lb = " + interval2.lb +  ", ub = " + interval2.ub);

        // 第三种情况实例
        double x3 = 2.45;
        Intervals interval3 = advanceAndRetreatMethod(x3, step);
        System.out.println("S2: x0 = " + x3 + ", lb = " + interval3.lb +  ", ub = " + interval3.ub);

    }

    // 进退法：确定搜索区间
    private static Intervals advanceAndRetreatMethod(double x, double h) {
    
    

        double lb = 0, ub = 0;

        if (Math.abs(func(x) - func(x+h)) <= 1e-6) {
    
    
            // 第三种情况
            lb = x;
            ub = x + h;
        }
        else if (func(x) < func(x+h)) {
    
    
            // 第一种情况
            ub = x + h;
            int lamb = 1;
            while (func(x-lamb * h) < func(x)) {
    
    
                lamb += 1;
            }
            lb = x - lamb * h;
        }
        else {
    
    
            // 第二种情况
            lb = x + h;
            int lamb = 2;
            while (func(x+lamb * h) < func(x + h)) {
    
    
                lamb += 1;
            }
            ub = x + lamb * h;
        }

        // 构造搜索区间对象
        Intervals interval = new Intervals();
        interval.lb = lb;
        interval.ub = ub;

        return interval;

    }

    // 待优化函数
    private static double func(double t) {
    
    
        return t * t - t * 5 + 8;
    }

    // 搜索区间对象
    private static class Intervals {
    
    
        double lb;
        double ub;
    }

}

After running the above code, you can get

S1: x0 = 2.0, lb = 2.1, ub = 2.9
S2: x0 = 4.0, lb = 1.0, ub = 4.1
S2: x0 = 2.45, lb = 2.45, ub = 2.5500000000000003