Cutting plane method and branch and bound method for solving integer programming problems

integer programming

Integer programming problems are linear or nonlinear programming problems in which the optimization variables must take on integer values, however, in most cases, integer programming problems are referred to as integer linear programming problems.

其数学模型为
m i n f ( x ) = c T x s.t A x = b x ≥ 0 x i ∈ I , i ∈ I ⊂ { 1 , 2 , . . . , n } min \quad f(\pmb x)=\pmb c^T\pmb x \\ \text{s.t} \quad \pmb A\pmb x=\pmb b \\ \pmb x ≥ 0\\ x_i \in I, i\in I \subset\{1,2,...,n\} minf(x)=cTxs.tAx=bx≥0xi​∈I,i∈I⊂{ 1,2,...,n }
Specially, ifI = { 0 , 1 } I = \{0, 1\}I={ 0,1 } , the above model is also called the 0-1 planning problem.

Compared with the linear programming problem that has been introduced before , the integer programming problem actually has an additional set of integer constraints. In view of such a close relationship between the two, the linear programming problem shown below is called a relaxation of the integer programming problem.
$$\begin{gathered} minf(x)=c^{T}x \\ \text{s.t}Ax=b \\ x\ge 0 \end{gathered}$$

Although it seems that the optimization variables are only subject to a set of integer conditions, in theory, the solution of integer programming problems is no longer polynomial complexity.

Currently there are two most commonly used algorithms for solving integer programming problems: cutting plane method and branch and bound method. Don’t be scared by the name, their essence is just to add some additional algorithmic logic to the simplex method to ensure that integer solutions can be obtained. These algorithm logics are more like algorithm frameworks, and the design ideas behind them can be clearly described through simple examples.

cutting plane method

This section explains the algorithm principle of the cutting plane method by solving the following integer programming problem.
$$\begin{gathered} minz=-5x{\_}_1-8x{\_}_2 \\ \text{s.t}{x}_1+x_2+x_3=6 \\ 5x{\_}_1+9x{\_}_2+x_4=45 \\ {x}_1,x_2,x_3,x_4\ge 0,And\; can\; only\; take\; integers. \end{gathered}$$
First calculate the corresponding relaxation problem and get the optimal solution as
$$x_1=9/4,x_2=15/4$$
In the figure below, point A is the optimal solution. Obviously, this solution does not satisfy the constraint that the optimization variables are integers.

At this time, the idea of ​​the cutting plane method is to first cut off the non-integer area near point A from the feasible domain, and then recalculate the optimal solution. The mathematical description of "cut" can be expressed as: adding a constraint to the relaxed problem. In this example, the constraint expression is
$$0.75x_3+0.25x_4\ge After\; adding\; constraints\; to\; 0.75$$
, the feasible region is as shown in the figure below. The optimal solution obtained by re-resolving is
$$x_1=0,x_2=5Although$$
this solution is the optimal solution obtained by solving the relaxation problem, since it also satisfies the constraints of the integer condition, it is naturally the optimal solution of the original integer programming.

The only question now is: how to get the new constraint expression? This is explained in detail next.

Before cutting, the simplex table corresponding to the optimal solution is as follows. The simplex table is obtained based on the simplex method, and this article gives a detailed explanation. This article does not dedicate a separate chapter to describe the creation and iteration process of the simplex table, mainly because these are not needed in practical applications.

			-5	-8	0	0
C_b	base	b	x_1	x_2	x_3	x_4
-5	x_1	9/4	1	0	9/4	-1/4
-8	x_2	15/4	0	1	-5/4	1/4

x 2 x_2 from simplex table$x_2$From that line, we can see that
$$\frac{15}{4}=0x{\_}_1+1x_2-\frac{5}{4}{x}_3+\frac{1}{4}{x}_4$$
Split the integer part and decimal part of the coefficient, we can get
$$3+\frac{3}{4}=(0+0)x_1+(1+0)x_2+(-2+\frac{3}{4})x_3+(0+\frac{1}{4})x_4$$

Combine the integer and fractional parts
$$(0x{\_}_1+1x_2-2x{\_}_3+0x{\_}_4-3)+(0x{\_}_1+0x{\_}_2+\frac{3}{4}{x}_3+\frac{1}{4}{x}_4)=\frac{3}{4}$$

The first term on the left side of the equation is the integer part, and the right side of the equation is $[0,1]$ is a decimal, so the decimal part of the second term on the left side of the equation must be greater than or equal to the value on the right side, that is
$$(0x{\_}_1+0x{\_}_2+\frac{3}{4}{x}_3+\frac{1}{4}{x}_4)\ge \frac{3}{4}$$
This formula is the constraint we just added.

Of course, as can be seen from the picture, the horizontal and vertical coordinates are $x_1,x_2$, but the constraints are about $x_3,x_4$, so for visualization, you need to convert
$$x_3=6-x_1-x_2,x_4=45-5x{\_}_1-9x{\_}_2$$
Then substitute the newly added constraints and become
$$2x{\_}_1+3x{\_}_2\le 15In$$
this way, you can draw the picture shown above.

As for why you should choose $x_2$This is mainly because experience shows that using the row with the largest decimal part to construct the constraints will result in faster convergence.

branch and bound

Compared with the cutting plane method, the idea of ​​​​the branch and bound method is easier to understand.

Take the following example as an example:
$$\begin{gathered} minf(x)=-10x_1-20x_2 \\ \text{s.t}5x{\_}_1+8x{\_}_2\le 60 \\ {x}_1\le 8 \\ {x}_2\le 4 \\ {x}_1,x_2\ge 0,And\; can\; only\; take\; integers \end{gathered}$$

(1) Define P as the original integer programming problem, P0 as its corresponding relaxation problem, the optimal solution is
$$x_0=(5.6,4),f_0=-136$$
due to$x_0$The integer constraint is not satisfied, so this solution is not the optimal solution of P. But the optimal solution of P $f^{\ast }$ will definitely not be lower than the optimal solution of P0, so$f_0$Can be used as the lower bound of P
$$f_{lb}=-136$$

In addition, we can easily find that $x=(0,0)$ is a feasible solution of P, at this time$f=0$ , the optimal solution of P$f^{\ast }$ will not be higher than this value, so the upper bound of P is
$$f_{ub}=0$$

(2) In the optimal solution of P0, since $x_1=5.6$ , introduce two mutually exclusive constraints:
$$x_1\le 5,x_1\ge 6$$
Add these two constraints to P respectively to obtain sub-problems P1 and P2. Obviously, the optimal solution of P is the same as the smaller of the optimal solutions of P1 and P2.

Solving the relaxation problem corresponding to P1, the optimal solution is
$$x_1=(5,4),f_1=-130$$
due to$x_1$is an integer solution, so it is also the optimal solution of P1, the upper bound $f_{ub}$It can be modified as
$$f_{ub}=f_1=-130$$
Since P1 has obtained the integer optimal solution, P1 does not need to continue to branch.

Solving the relaxation problem corresponding to P2, the optimal solution is
$$x_2=(6,3.75),f_2=-135$$
$x_2$The integer condition is not satisfied, so it is not the optimal solution for P2, but $f^{\ast }$ will not be lower than$f_2$, so the lower bound
$$f_{lb}=-135$$

(3) In the optimal solution of P2, since $x_2=3.75$ , continue to introduce two mutually exclusive constraints
$$x_2\le 3,x_2\ge 4$$
Add these two constraints to P2 respectively to obtain sub-problems P3 and P4.

First solve the relaxation problem corresponding to P4. There is no feasible solution, so the branching can be stopped.

Then solve the relaxation problem corresponding to P3, the optimal solution is
$$x_3=(7.2,3),f_3=-132$$
$x_3$does not satisfy the integer condition, so it is not the optimal solution of P3, but $f^{\ast }$ will not be lower than$f_3$, so you can continue to update the lower bound
$$f_{lb}=-132$$

(4) In the optimal solution of P3, since $x_1=7.2$ , continue to introduce two mutually exclusive constraints
$$x_1\le 7,x_1\ge 8$$
Add these two constraints to P3 respectively to obtain sub-problems P5 and P6.

Solving the relaxation problem corresponding to P5, the optimal solution is
$$x_5=(7,3),f_5=-130$$
due to$x_5$is an integer solution, so it is also the optimal solution of P5, the upper bound $f_{ub}$It can be modified as
$$\overline{f}=f_5=-130At$$
this time, P5 does not need to be branched anymore.

Solving the relaxation problem corresponding to P6, the optimal solution is
$$x_6=(8,2.5),f_6=-130$$
$x_6$The integer condition is not met, but $f_6$Not less than the current upper bound $f_{ub}$, so the branch is a "dead branch" and needs to be pruned.

Combining P5 and P6, the lower bound can be updated as
$$f_{lb}=-130$$

At this time, we find
$$f_{ub}=f_{lb}=-130So$$
the optimal solution to this problem is
$$x_1=(5,4)or{x}_5=(7,3)$$
The corresponding objective function value is
$$f^{\ast }=-130$$

The whole process of branch and bound can be referred to the figure below.

In general, both the cutting plane method and the branch-and-bound method first calculate the relaxation problem corresponding to the original problem, and then determine whether the optimal solution to the relaxed problem also satisfies the integer constraints. If so, then everyone is happy; otherwise, the cutting plane method will By adding constraints, the feasible region of the relaxed problem is improved in order to achieve the goal that the optimal solution to the relaxed problem is also the optimal solution to the original problem; the branch-and-bound rule uses decomposition technology to decompose the original problem into several sub-problems and calculate them separately. , and then continue to update the upper and lower bounds of the original problem based on the solution results of the sub-problem until they are equal.

Code

Although the cutting plane method and the branch and bound method seem to have a lot of steps, fortunately, the solver has already done the integration work for us, so we can directly call the ready-made solver to solve the integer programming we encounter. question.

The only difference between the code based on Python calling ortools to solve the integer programming problem and the linear programming code introduced before is that the definition of optimization variables in integer programming is solver.IntVar, while the definition method in linear programming is solver.NumVar.

The following is the solution code for the integer programming problem in the previous section.

from ortools.linear_solver import pywraplp


if __name__ == '__main__':
    # 声明ortools求解器，使用SCIP算法
    solver = pywraplp.Solver.CreateSolver('SCIP')

    # 优化变量
    x1 = solver.IntVar(0, 8, 'x1')
    x2 = solver.IntVar(0, 4., 'x2')

    # 目标函数
    solver.Minimize(-10 * x1 - 20 * x2)

    # 约束条件
    solver.Add(5 * x1 + 8 * x2 <= 60)

    # 模型求解
    status = solver.Solve()

    # 模型求解成功, 打印结果
    if status == pywraplp.Solver.OPTIMAL:
        # 变量最优解
        print('x1: {}, x2: {}'.format(x1.solution_value(), x2.solution_value()))

        # 最优目标函数值
        print('best_f =', solver.Objective().Value())

    else:
        print('not converge.')

After running the code, the optimal solution can be obtained as follows. Obviously, this solution is consistent with the result deduced in the previous section.

x1: 5.0, x2: 4.0
best_f = -129.99999999999997