So far, the optimization algorithms we have introduced all start from the whole and search for the global optimum. And most of them start from an initial feasible solution (branch and bound search is arguably not in a sense). In other words, these algorithms are direct solvers. But some problems are too complex, and there are too many variables to optimize, and the scale is so large that it is impossible to start from the overall consideration. Today we introduce an indirect solution algorithm, which decomposes the original problem into multiple subproblems that are simple enough to be solved directly by iteration, and the accompanying master problem combines the results of all subproblems to give the exact or near optimal solution. Without further ado, let's get to the point.
1
column generation algorithm
The first thing to introduce is the column generation algorithm, which we mentioned when we introduced the classification of discrete optimization models (set packing, covering, partitioning models).
Portal: Discrete Optimization Models
Column generation algorithms are suitable for optimization problems that satisfy the following characteristics:
The corresponding mathematical model of the problem can simply use every possible column of the constraint matrix to represent the choice of decision variables , these feasible columns are generated by an independent subproblem construction, and can satisfy even those complex Restrictions.
The idea of the column generation algorithm is to achieve global optimization by iteratively solving the partial master problem based on the given generated columns —the main problem here refers to the restricted original problem (restricted version), which only includes Columns corresponding to decision variables of zero. If the original problem is modeled as an integer linear programming problem, then a linear relaxation of the restricted problem can be solved. The mathematical expression of its algorithm is:
We give the general steps of the column generation algorithm:
The dual problem was introduced earlier and in detail, Portal: A Dual Theory of Linear Programming
The most difficult part of this algorithm is how to construct a test number less than 0 in step 2. One way is to take out all the columns of J and try them one by one, but the efficiency is undoubtedly low. We can adopt a heuristic algorithm, as long as it is judged whether the minimum number of tests in all feasible columns is less than 0.
We use a case to understand the algorithm - furniture profile cutting.
The profiles required for furniture manufacturing are all cut from the feed material with a length of b=11 meters. Now the profiles of different lengths are required as shown in the table below. How should the manufacturer cut the materials to meet the demand and save the most materials at the same time? ?
First, a cutting scheme k is defined as a combination of the number of cuts (aik) of profiles of different lengths (pi) with a total length not exceeding b meters. Let K represent the set of all possible cutting schemes, and xk represent the number of times cutting scheme k is used, then the blanking problem can be described as the following integer programming problem:
We need to understand here that plan k is to cut profiles with a length of no more than 11 meters and cut them into different combinations, such as some combinations shown below:
The big K represents a set of cutting schemes that meet the requirements. For example, one of its elements can be :
The above element is also the easiest cutting scheme to construct the initial restricted master problem (four schemes that only cut single-length profiles). According to algorithm step 2, the optimal dual solution {vi} can be obtained by solving the linear relaxation problem of the current restricted master problem l of the model, then the next generation column p can be obtained by solving the knapsack problem:
Below we implement the column generation algorithm. First, use the simplest cutting scheme to get the initial master limit problem:
Get the optimal solution (the target value is 32):
Then go to step 2 and construct the knapsack problem:
Get the optimal solution:
Add a new cutting scheme:
Going back to step 1, solve a new planning problem:
The optimal solution is (target value 25.13):
Then go to step 2 and construct the knapsack problem:
Get the optimal solution (the target value is less than 0):
Add new cutting scheme:
Go back to step 1, solve the new planning problem, and get the optimal solution (target value 24.83):
Go to step 2, construct the knapsack problem, and get the optimal solution (target value -1):
Add new cutting scheme:
Go back to step 1 and recalculate the planning problem, and the optimal solution remains unchanged at this time. No new decision columns are added. So we get an approximate solution:
It can be seen that the column generation algorithm is suitable for solving a class of combinatorial optimization problems in which candidate decision-making schemes can be represented by multiple columns under complete information. The algorithm does not directly solve the planning problem with all columns, but solves the model containing only the currently generated columns by limiting the main problem, and then selects the columns that may improve the objective function value of the main problem by the column generation sub-problem. And added to limit the main problem. The advantage of this algorithm is that in the end the columns that are not generated by the subproblems are judged as not improving the current optimal solution of the constrained main problem, thus avoiding considering a large number of possible columns.
When it comes to the case of household profile cutting, it is not necessary to list all possible cutting schemes in advance. Column generation algorithms are important ideas for solving large-scale optimization problems, for a better understanding. Let us give another case— plan optimization for intensity-modulated radiation therapy (IMRT).
The goal of radiation-based therapy is to eliminate the tumor with the strongest radiation, but it also needs to minimize the radiation to surrounding normal tissues to avoid damage to them. The equipment used in radiotherapy is a large accelerator, which can output ray beams from different angles around the human body, so that not only can a higher radiation dose be applied to the tumor lesion target area, but also the surrounding normal tissues can be less irradiated. Since the radiation beam output by the accelerator is generally large, intensity modulated radiation therapy (IMRT) combines many sub-beams with different irradiation intensities to form a radiation beam to improve the accuracy of treatment planning.
The sub-beams are virtual and do not physically exist. They confine the radiation dose in an open aperture by means of a multi-leaf collimator with moving blades, and focus to produce radiation effects (as shown in the figure below). The concentrated radiation effects of the different beamlets are superimposed like a bar graph and finally emitted at different angles.
Not every combination of seed bundles forms an effective aperture. Under some constraints, an effective gap can only be created by moving the blade between the two boundaries; if the "opening" is closed or in the middle area like row 2 of Fig. b, it is impossible to create beamlets. If there is overlap of blades similar to the 2nd and 3rd rows in Figure c, it will cause a "collision risk" and cannot form an effective gap.
Each organ tissue is represented by a discrete point set. The goal of radiotherapy plan optimization is to maximize the total radiation dose of the lesion target area (or the average radiation dose of each point), and the constraint condition is that the total radiation dose received by each point of healthy tissue Radiation doses must not exceed safe levels. The input parameters estimate the beamlet radiation dose to each point on tissue per unit intensity:
The following table shows an example containing 2 ray beams, 18 sub-beams, and 12 points. The safety levels of the highest radiation doses at 3 healthy tissues are b={50,65,70}
Once the set of beamlets Qm,k from aperture m at ray beam k is selected, then:
From this, the linear programming model of this case can be obtained:
The initial solution leaves all beamlets of the two ray beams turned on.
2
Branch Pricing Algorithm
It can be seen that using the column generation algorithm often keeps both integer and decimal solutions in the column generation sub-problems, and we can easily obtain an approximate optimal solution to the main problem by rounding the optimal solution of the linear relaxation problem corresponding to the restricted main problem. However, in many practical problems, since the decision variable is an integer variable of 0-1, the solution of the restricted master problem will become very complicated, and even rounding cannot guarantee an approximate optimal solution . Therefore, we need to use the enumeration idea of branch and bound search, combined with the idea of column generation algorithm, to get a branch pricing algorithm (branch and price) to solve this problem:
Let's understand this algorithm through an example:
Assuming that the original problem is a 0-1 integer programming to minimize the objective, during initialization, four columns are selected to constitute the constraints of the programming problem, and the root node and the initial solution x(0) are obtained, which happens to be an integer solution, and the optimal solution and The corresponding optimal solution is updated.
Then repeat steps 2 to 4 twice below, add column 5 for the first time, and get the solution x(1) of the linear relaxation problem, if the integer constraint of 0-1 is not satisfied, go to step 4, choose to add column 6 again, and get a new Solve x(2), at this time, no new column j that satisfies the condition can be found. Since the optimal solution 124.56 is smaller than the current optimal solution 136, this branch cannot be stopped immediately. Branching is performed, assuming that the component 2 is rounded to 0, and the node 3 no longer satisfies the integer constraint, so it is not feasible to terminate. The optimal solution of branch 4 rounded to 1 is x(4), and the optimal solution value is 130.55. At this time, column 7 is added to obtain the optimal solution of node 5. Finally, branch 6 is obtained (the integer component is 7), and the optimal solution satisfies the 0-1 integer programming. So terminate the branch.