Primal Dual Mode

primordial dual mode

background review

Over the past forty years, combinatorial optimization has been strongly influenced by linear programming. With the mathematical and algorithmic understanding of linear programming came a wealth of ideas and tools that were then applied to combinatorial optimization. Many of these ideas and tools are still in use today and form the basis of our understanding of combinatorial optimization. With the help of the dual theory of linear programming, a new technique for designing approximation algorithms - the primal dual pattern (PD) - was proposed.

The Primal-Dual Method originated from the design of precise algorithms. In this case, this pattern yields the most efficient algorithms for some fundamental problems in P, including matching, network flows, and shortest paths. A property of these problems is that their LP relaxations have integer optimal solutions. For example, in Huang Hongxuan's "Mathematical Programming" [7], the design idea of ​​the original dual algorithm is to ensure that the dual is feasible, satisfy the complementary relaxation conditions, and gradually improve the original infeasibility. When they are both satisfied, both solutions are optimal. In the iterative process, integer corrections are always made to the original solution, and an integer optimal solution is finally obtained. The basic process of the algorithm is as follows:

With the primal dual mode (PD), when we require an approximate solution to an integer programming problem, after relaxing it, we do not have to find the optimal solution to the corresponding linear programming problem. As we will see, PD methods are very powerful. Usually, we can use the PD method to get a good approximation algorithm, and then extract a good combination algorithm from it. Instead, we can sometimes use PD methods to demonstrate good performance of combinatorial algorithms by simply reinterpreting them as PD algorithms. So far we have seen many algorithms based on linear programming (LP) relaxations, which usually involve combining the given The fractional LP solution rounds to approximately the same target value as the integer solution.

Basic introduction

Proposer of Duality Theory—John Von Nouma (John Von Nouma, 1903-1957), one of the scientific all-rounders in the fields of American-Hungarian mathematician, computer scientist, game theory, nuclear weapons and biological and chemical weapons, was Later generations are called "the father of modern computer" and "the father of game theory"

Dual theory has many important applications: when solving any one of the original and dual linear programs, the optimal solution of the other program is automatically given; when the dual problem has fewer constraints than the original problem, the solution The dual programming is much more convenient than solving the original programming; the variable in the dual programming is the shadow price, which can provide useful information for enterprise management decision-making.

We start with a general covering LP for duality theory. Suppose we have matrix A and vector $c\in R^n$，$b\in R^m$ . We can express the original LP as

Now suppose we want to determine a lower bound on the optimal value of this LP. One way to do this is for some $Z$ , using the constraints in LP, finds that "looks" like${\sum }_j{c}_j{x}_j\ge Z$ constraints. To do this, note that any convex combination of constraints from LP is also a valid constraint. Therefore, if we have non-negative multipliers$y_i$, we get a new constraint that is satisfied by all feasible solutions of the original LP, namely if for all $i$，${\sum }_j{a}_{ij}{x}_j\ge b_i$有
$$\sum _i{y}_i(\sum _j{a}_{ij}{x}_j)\ge \sum _i{y}_i{b}_i\left(1\right)$$
Note that we require$y_i$is nonnegative because multiplying an inequality by a negative number changes the sign of the inequality. (If the constraint is of the form ${\sum }_j{a}_{ij}{x}_j=b_i$, then its multiplier $y_i$can be any real number). Considering formula (1), if we ensure that ${\sum }_i{y}_i({\sum }_j{a}_{ij}{x}_j)\le {\sum }_j{c}_j{x}_j$We will get a lower bound of the original LP optimal value, namely
$$\sum _i{y}_i{b}_i\le \sum _i{y}_i\left(\sum _j{a}_{ij}{x}_j\right)\le \sum _j{c}_j{x}_j$$
Changed order, we got to
$$\sum _i{y}_i{b}_i\le \sum _i{y}_i(\sum _j{a}_{ij}{x}_j)=\sum _j(\sum _i{y}_i{a}_{ij})x_j\le \sum _j{c}_j{x}_j\left(2\right)$$
and can be obtained by requiring$y_i$The following conditions are met to ensure that the sum is at most
$$\sum _i{y}_i{a}_{ij}\le c_j\forall j=1,...,n\left(3\right)$$
Note that in the previous step we depended on$x_j$is non-negative, and at the same time, $y_i$The constraints on are linear, and we get a lower bound that is also linear. This way we can write an LP to find the optimal lower bound. It is possible to write a dual program (DLP):

This can be expressed in more compact notation:

Weak duality : for any feasible primal dual solution $(x,y)$ ,if
$${\mathbf{y}}^{\top }b\le {\mathbf{c}}^{\top }x$$

The proof is available from formula (2).

Strong duality : If either the primal or the dual have bounded optimal solutions, then they both have. And their objective value is equal. That is, if $x$ is optimal for the original,$y$ is optimal for dual, then
$$y^{\top }b=c^{\top }x$$

We generally don't prove the strong duality theorem, however, one way to prove it is to look at equation (2) and note that if $y^{\top }b=c^{\top }x$, then the inequality must be strict, someone might say, if$y^{\top }b\le c^{\top }x$, then we must be able to$x$或$y$ to make some improvements. Strong duality gives the optimal$(x,y)$ pairs, namely the following equations, which are obtained by combining equation (2) with${\mathbf{y}}^{\top }b={\mathbf{c}}^{\top }x$combination and achievable.
$$\sum _i{y}_i{b}_i=\sum _i{y}_i(\sum _j{a}_{ij}{x}_j)$$
$$\sum _j(\sum _i{y}_i{a}_{ij})x_j=\sum _j{c}_j{x}_j$$
Combining these equations with the constraints in primal and dual linear programming, we get complementary relaxation conditions.

Complementary relaxation : let $(x,y)$ is the solution of a pair of primal dual linear programming with a bounded optimal solution, then$x$和$y$ are optimal if and only if the following conditions hold:
the original complementary relaxation condition:
$$\forall j=1,...,n,x_j=0or\sum _i{y}_i{a}_{ij}=c_j$$

Dual complementary relaxation conditions :
$$\forall i=1,...,m,y_i=0or\sum _j{x}_j{a}_{ij}=b_i$$
When designing an approximation algorithm based on linear programming, first of all, we put a minimum (or maximum) problem $\Pi$ relaxation is a linear program${Pi}_{LP}$. Then, we solve the linear programming problem ${Pi}_{LP}$An optimal solution of $OP{T}_{LP}$, and rounding it becomes the problem Π \PiA feasible solution for$\Pi \; .$Note that the optimal solution to the linear programming problem$OP{T}_{LP}$The objective value of gives the problem Π\PiA lower bound (or upper bound) of the optimal solution OPT of$\Pi \; ,\; we\; often\; use\; the\; difference\; between\; these\; two\; values\; to\; estimate\; the\; performance\; ratio\; of\; this\; approximate\; algorithm.$Now according to the dual theory, each dual feasible solution provides us with the linear programming problem${Pi}_{LP}$A lower bound (or upper bound) of the optimal value, therefore, it is also the original problem Π \PiA lower bound (or upper bound) of the optimal value OPT of$\Pi \; .$This means that a "good enough" dual feasible solution can also be used to determine the performance ratio of the approximation algorithm. Therefore, we don't have to solve the linear programming problem${Pi}_{LP}$The optimal solution of $OP{T}_{LP}$And the corresponding target value, but only need to find a "good enough" dual feasible solution, and then transform it into the original problem $A\; feasible\; solution\; of\; \Pi$ , and finally use the difference between the target values ​​of these two solutions to estimate the performance ratio of the approximation algorithm.

primordial dual mode

The primal-dual schema is the preferred method for designing approximation algorithms because it usually gives algorithms with good approximation guarantees and good running times. The basic idea is to deal with the NP-hard problem and its dual LP relaxation problem, and then iteratively transform the original solution and the dual solution until the relaxed primal dual complementary relaxation condition is satisfied.
Relaxed original complementary relaxation conditions :
$$\forall j=1,...,n,x_j=0or\frac{c_j}{a}\le \sum _i{y}_i{a}_{ij}\le c_j$$
Relaxed dual complement relaxation conditions :
$$\forall i=1,...,m,y_i=0or{b}_i\le \sum _j{x}_j{a}_{ij}\le b\cdot b_i$$

If x and y are the primal feasible solution and the dual feasible solution respectively, satisfying the above relaxation complementary conditions, then
$$\sum _j{c}_j{x}_j\le ab\sum _i{y}_i{b}_i$$

Definition:
$$\begin{gathered} \sum _j{c}_j{x}_j\le a\sum _j(\sum _i{y}_i{a}_{ij})x_j \\ =a\sum _i{y}_i(\sum _j{a}_{ij}{x}_j)\le ab\sum _i{y}_i{b}_i \end{gathered}$$
Lemma 1 shows that the approximation guarantee of the algorithm is $ab$ .

Integral Gap for Linear Programming Relaxation

Given the minimization problem $Linear\; programming\; relaxation\; of\; \Pi$ , with$OP{T}_f\left(I\right)$ represents the cost of the optimal fractional solution of instance I, that is, the objective function value of the linear programming relaxation optimal solution. Then the relaxed integer gap (sometimes called integer ratio) is
$$\underset{I}{\sup }\frac{OPT(I)}{OP{T}_f(I)}$$
That is, the supremum of the ratio of the optimal integer solution and the optimal fractional solution.

For the maximization problem, the integral gap is defined as the infimum of this ratio. When the integral gap of the linear program is 1, we call this linear programming relaxation an exact relaxation. If the cost of the solution found by the algorithm is directly compared with the cost of the optimal fractional solution (or dual feasible solution), then we can expect that the best approximation factor for the proof should be the relaxed integer gap. Interestingly, primal-dual techniques have succeeded in obtaining algorithms that approximate the integer gap that is guaranteed to be exactly equal to the slack.

Approximation Algorithms for Primal Dual Patterns

Next, we introduce the approximation algorithm based on the primal dual mode, as follows:

The advantage of the primal-dual mode is that it avoids a lot of time spent in finding the optimal solution of the primal linear programming problem, thereby reducing the running time of the algorithm. In particular, the current time complexity of the fastest interior point algorithm for solving linear programs is also $O\left(n^{3.5}\right)$magnitude. Moreover, in some practical applications, what needs to be solved is an online problem. At this time, it is not realistic to ask for the optimal solution of the linear programming problem. The acceleration effect brought by the primal dual mode is necessary.

Summarize

We mainly introduce the basic content of the primal-dual technique, and then show how the primal-dual technique is modified for different problems in order to provide good approximation algorithms for various NP-hard problems. Stay tuned!

references

[1] R. Bar-Yehuda and S. Even. A linear time approximation algorithm for the weighted vertex
cover problem. Journal of Algorithms, 2:198–203, 1981.
[2] Michel X. Goemans and David P. Williamson. The primal-dual method for approximation algorithms and its application to network design problems. In Dorit Hochbaum, editor, Approximation algorithms for NP-hard problems, chapter 4, pages 144–191. PWS Publishing Co., Boston, MA, USA, 1997.
[3] D. Williamson. The primal-dual method for approximation algorithms. Mathematical Programming, Series B, 91(3):447–478, 2002.
[4] H.W. Kuhn. The Hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2:83–97, 1955.
[5] GB Dantzig, LR Ford, and DR Fulkerson. A primal–dual algorithm for linear programs. In HW Kuhn and AW Tucker, editors, Linear Inequalities and Related Systems, pages 171–181. Princeton University Press, Princeton, NJ, 1956.
[6] MX Goemans and DP Williamson. The primal–dual method for approximation algorithms and its applications to network design problems. In DS
[7] Huang Hongxuan. Mathematical Planning: Tsinghua University Postgraduate Public Class Textbook. Mathematics Series [M]: Tsinghua University Press, 2006.