Integer Programming - Chapter 2 Linear Programming

Integer Programming - Chapter 2 Linear Programming

2.1 Preliminary convex analysis

2.1.1 Convex sets and separation theorem

Definition 2.1 Let the set $C\subseteq {\mathbb{R}}^n$ , if for any$x,y\in C$ 和 $l\in [0,1]$，有
$$\lambda x+(1-l)y\in C,$$

is called $C$ isa convex set.

In convex geometry, a convex set is a subset of the affine space closed under convex combinations. More specifically, in Euclidean space, a convex set is such that for every pair of points in the set, every point on the straight line segment connecting the pair of points is also in the set. For example, a cube is a convex set, but anything that is hollow or has indentations such as a crescent is not a convex set.

Theorem 2.1 Suppose $C\subseteq R^n$ is a non-empty closed convex set,$y\in R^n,y\notin C$ , then

1. There is a unique point $\overline{x}\in C$，使
   ∣ ∣ x ‾ − y ∣ ∣ = inf ⁡ { ∣ ∣ x − y ∣ ∣   ∣   x ∈ C } ||\overline{x}-y|| = \inf \{||x-y||\ |\ x\in C \} ∣∣x−y∣∣=inf{ ∣∣x−y∣∣ ∣ x∈C}

   In mathematical notation, "inf" stands for "infimum", which is a concept in mathematical analysis and set theory. The infimum of a set is the largest real number in the lower bound of the set. Simply put, it is the smallest real number that is less than or equal to all the numbers in the set.

   This theorem shows that there exists a non-empty closed convex set with a given point $y$ is the closest point, and this point is unique.

2. $\overline{x}\in C$ is point$y$ to setCCThe necessary and sufficient condition for the nearest point of$C$
   $$(x-\overline{x}{)}^T(\overline{x}-y)\ge 0,\forall x\in C\text{\hspace{1em}(2.1)}$$

Theorem 2.2 Convex Set Separation Theorem设$C\subseteq {\mathbb{R}}^n$ is a non-empty closed convex set,$y\in R^n,y\notin C$ , then there is a non-zero vector$a\in R^n$ and number$\beta$ 使得
$$a^{\text{T}}x\le b<a^{\text{T}}y,\forall x\in Specifically$$
:

• $a$ 是一个非零向量，它定义了一个超平面。
• $\beta$ 是一个实数，它表示超平面在 $a^{\text{T}}x$ 方向上的偏移量。
• 对于集合 $C$ 中的所有点 $x$，它们都位于这个超平面的一侧，并且超平面分离了点 $y$ 和集合 $C$ 。

这个定理直观地说明了在凸集 $C$ 和点 $y$ 之间存在一个超平面，将点 $y$ 与集合 $C$ 完全分开，即点 $y$ 在超平面的一侧，而集合 在另一侧 $C$ 。这个超平面可以看作是一条线（$n=2$ ）或者一个平面（$n=3$）或者更高维空间中的超平面。

2.1.2 多面体基本知识

Definition 2.2 If $S\subseteq R^{If\; n}$ is the intersection of a finite number of half spaces, it is called$S$ 为多面体，即 S = { x ∈ R n ∣ a i T ≤ b i ,   i = 1 , . . . , m } S=\{x\in \R^n |a^T_i \le b_i,\ i=1,...,m\} S={ x∈Rn∣aiT​≤bi​, i=1,...,m } ,$a\in R^n,ai\ne 0，b_i\in R$

It is easy to see that polyhedra are closed convex sets . Let $A\in R^{m\times n},b\in R^m$ , the following sets are all polyhedrons:
S = { x ∈ R n ∣ A x ≤ b } , S = { x ∈ R n ∣ A x = b , x ≥ 0 } , S = { x ∈ R n ∣ A x ≤ b , x ≥ 0 } . \begin{aligned} &S=\{x\in \R^n| Ax\le b\},\\ &S=\{x\in \R^n| Ax= b ,\ x\ge 0\},\\ &S=\{x\in \R^n| Ax\le b,\ x\ge 0\}. \end{aligned}​S={ x∈Rn∣Ax≤b},S={ x∈Rn∣Ax=b, x≥0},S={ x∈Rn∣Ax≤b, x≥0}.​

A polyhedron is a set of solutions to a (finite number of) linear equations and inequalities.

A half-space is the region formed by a hyperplane and all its points on a specific side

Definition 2.3 Let $S\subseteq R^n$ is a non-empty convex set,$x\in S.$ _ If for any$l\in (0,1)$ and$x_1,x_2\in S$ ,$x=\lambda x_{1}ten(1-\lambda )x_2$It can be deduced that $x_1=x_2=x$ , then it is called$x$ isSSThe vertex of$S.$

That is if CCThere are no two different points in$C$$x$ becomes a point on the line connecting these two points, then$x$ is the vertex.

Definition 2.4 Let $S\subseteq {\mathbb{R}}^n$ is a non-empty convex set,$d\in R^n$ . If for any$x\in S$ sum$l\ge 0$ has$x+\lambda d\in S$ , then it is called$d$ isSSA direction of$S.$If for any$l_1,l_2>0$ andSSThe direction of$S$$d_1,d_2$, $d=l_1{d}_1+l_2{d}_2$It can be deduced that $d_1=\alpha d_2$, where $a>0$ , then it is called$d$ isSSThepolar directionof$S.$(If$S$ direction$d$ 不能表示为该集合的两个不同方向的正的线性组合，则称 $d$ 为 $S$ 的极方向。)

直观上来说，极方向可以看作是在凸集 $S$ 的边界点 $x_0$ 处的“最远延伸方向”。在极方向上，你可以在该方向上无限远地向外延伸，而仍然保持在凸集 $S$ 内部。

定理2.3 设 S = { x ∈ R n ∣ A x = b , x ≥ 0 } S=\{x\in \R^n|Ax=b,x\ge0\} S={ x∈Rn∣Ax=b,x≥0}，其中 $A$ 行满秩，则 $x\in S$ 是 $S$ 的顶点的充要条件为$x$ can be expressed as $ x = \left( \begin{aligned} &B^{-1}b \cr &0 \end{aligned} \right)$, where A = ( B , N ) A=(B,N$A=(B,N)$，$B$ is reversible and$B^{-1}b\ge 0$

定理2.4 设 S = { x ∈ R n ∣ A x = b , x ≥ 0 } S=\{x\in \R^n|Ax=b,x\ge 0\} S={ x∈Rn∣Ax=b,x≥0 } non-empty, where$Row\; A$ is of full rank, then$S$ has at least one vertex.

2.2 Linear programming and primitive simplex method

Consider the standard form of Linear Programming (LP):
( LP ) min ⁡ { c T x ∣ A x = b , x ≥ 0 } (LP)\quad \min\{c^{\text{T}} x|Ax = b,x\ge 0\}(LP)min{ cTx∣Ax=b,x≥0}
此处 $A\in R^{m\times n}$，$A$ 行满秩，设 S = { x ∈ R n ∣ A x = b , x ≥ 0 } S=\{x\in \R^n | Ax = b, x\ge 0\} S={ x∈Rn∣Ax=b,x≥0 } , called$S$ is the feasible region of (LP).

In solving a system of linear equations, a full-rank matrix often has a unique solution.

Theorem 2.7 (Basic Theorem of Linear Programming) Let the linear programming feasible region $S$ is not empty,$The\; vertices\; of\; S$ are$x_1,...,x_k$, the polar direction is $d_i,...,d_l$, then the necessary and sufficient conditions for (LP) to have a finite optimal solution are $c^T{d}_i\ge 0，i=1,...,l$。If (LP) has a finite optimal solution, then (LP) must be at one of the vertices $x_j$reach the optimal.

Definition 2.5 Consider the feasible region $S$，设$A=(B,N)$ , where$If\; B$ is reversible, it is called$BAAA$ _$A$ 的一个基，$x=\left(\begin{array}{rl}& x_B\\ & x_N\end{array}\right)=\left(\begin{array}{rl}& B^{-1}b\\ & 0\end{array}\right)$为$Ax=$Basic solutionof$b$ ,$x_B$are called base variables , $x_N$is a non-basic variable. If $B^{-1}b\ge 0$ , then it is called$B$ isthe original feasible basis,$x=\left(\begin{array}{rl}& B^{-1}b\\ & 0\end{array}\right)$ is calledthe basic feasible solution.

Known matrix $A$ 是 $m\times$matrix of$n$$A$ has full row rank (that is, every row of equations is meaningful), and$m<n$ , 则$The\; basis\; vector\; of\; A$ is$m$ . We rearrange Aby columnsand express it as$A=(B,N)$ , where$B$ by$m$ basis vector representation,$N$ is given by other$n-It\; consists\; of\; m$ non-basic vectors. Correspondingly,$x$ can also be expressed as a combination of basic variables and non-basic variables$x=\left(\begin{array}{rl}& x_B\\ & x_N\end{array}\right)$，目标函数表示为 $c_B^T{x}_B+c_N^T{x}_N$ 。对于等式 $Ax=b$ ，可以表示成 $B{x}_B+N{x}_N=b$ 。

非基变量$x_N$ 全部取值为0得到的解称之为基解，基解的个数最多为 $C_n^m$，若 $x>0$，则称之为基本可行解。

假设2.1（非退化假设） 对任意基本可行解$x=\left(\begin{array}{rl}& B^{-1}b\\ & 0\end{array}\right)$，都有 $B^{-1}b>0$

性质2.1 线性规划的可行域S的顶点和基本可行解一一对应，线性规划若有有限最优解，则一定存在基本可行解是最优解。

上述定理和性质表明，我们只需在可行域的顶点中寻找线性规划的最优解.

单纯形法

单纯形：n维空间内有n+1个顶点的多面体

算法2.1 原始单纯形算法

步0：计算初始基本可行基 $B$ 和对应的基本可行解 $x=\left(\begin{array}{rl}& B^{-1}b\\ & 0\end{array}\right)$。

步1：如果 $r_N=c_N^T-c_B^T{B}^{-1}N\ge 0$，停止，当前基本可行解 $x$ 是最优解，否则，转步2.

步2：选取 $j$ 满足 $c_j-c_B^T{B}^{-1}{a}_j<0$，若 ${\overline{a}}_j=B^{-1}{a}_j\le Stop\; at\; 0$ , the original problem is unbounded; otherwise, go to step 3.

Step 3: Calculate λ \lambda from the following formula$\lambda$，令$x:=x+\lambda d_j$，其中 $d_j=\left(\begin{array}{rl}& -B^{-1}{a}_j\\ & e_j\end{array}\right)$，转步1
$$l=\min \left\{\frac{{\overline{b}}_i}{{\overline{a}}_{ij}}|{\overline{a}}_{ij}>0\right\}=\frac{{\overline{b}}_r}{{\overline{a}}_{rj}}\ge 0$$

Internal idea: First we can determine a set of basis, and then find the basis feasible solution through this set of basis. This is the work of step 0-1. After we find the basic feasible solution, we still need to judge whether it is the optimal solution. This is the work optimality test of step 2. Suppose we know after testing that the solution we are looking for is the optimal solution, then we are really lucky. If not, it doesn't matter. We will enter the third step to change the base variable. In this way, a new set of basic feasible solutions can be found, and then optimality testing is performed until the optimal solution is found.

The simplex method is to move from one vertex to another better vertex on a simplex in any n-dimensional space. Until a better vertex cannot be found, it means that the optimal solution has been reached.

Theorem 2.8 Under the assumption of non-degeneracy, the original simplex algorithm terminates in finite steps, or finds an optimal solution of (LP), or determines that (LP) is unbounded.

The above algorithm can be expressed in the form of a table, assuming $b\ge 0$ , let the initial feasible basis be$B$ ，$A=(B,N)$ , then one iteration process of the simplex method can be performed on the following table:

"rhs" is the abbreviation of "right-hand side" (right-hand constant)

Example 2.1 Use the simplex algorithm to solve the following linear programming
$$\begin{array}{rl}& \min -7x{\_}_1-2x{\_}_2\\ & s.t.-x_1+2x{\_}_2+x_3=4,\\ & 5x{\_}_1+x_2+x_4=20,\\ & 2x{\_}_1+2x{\_}_2-x_5=7,\\ & x\ge 0\end{array}$$
The initial simplex table is as follows:

Iteration 1. Select the initial basis $B=(a_1,a_3,a_4)$，以 $x_B=(x_1,x_3,x_4{)}^{}$The simplex table with${}^T$$x_B=(x_1,x_3,x_4{)}^T=(3\frac{1}{2},7\frac{1}{2},2\frac{1}{2})，x_N=(x_2,x_4{)}^T=(0,0{)}^T$

The second iteration, because $-\frac{7}{2}<0$ , select$x_5$ 为进基变量，计算$\lambda =\frac{2\frac{1}{2}}{2\frac{1}{2}}=1$，故 $x_4$ 是离基变量，新的基变量为$x_B=(x_1,x_3,x_5)$。新的单纯形表为表2.3：

第3次迭代，选择 $x_2$ 为进基变量，计算$\min \left\{\frac{8}{11/5},\frac{4}{1/5}\right\}$。故 $x_3$ 是离基变量，新的单纯形表2.4，由于$r_N=(\frac{3}{11},\frac{6}{11})\ge 0$ , the current basic feasible solution$x=(\frac{36}{11},\frac{40}{11},0,0,\frac{75}{11}{)}^T$ is the optimal solution of linear programming, and the optimal value is$-30\frac{2}{11}$。

In the above simplex algorithm, we require that the problem has an initial basic feasible solution. Let's discuss how to find the initial feasible solution of the problem under general circumstances. Consider the standard form of linear programming:
( LP ) min ⁡ { c T x ∣ A x = b , x ≥ 0 } (LP)\quad \min\{c^{\text{T}}x|Ax = b, x\ge 0\}(LP)min{ cTx∣Ax=b,x≥0 }
Here$b\ge 0$ , introducing artificial variables$y\in {\mathbb{R}}_{+}^m$, consider the following linear programming:
$$(L{P}^a)\mathrm{my}\{e^{\text{T}}y|Ax+y=b,x\ge 0,y\ge 0\}$$
The following conclusions are obviously true:

• Linear Programming ( $L{P}^a$ ) Feasible and feasible solution$(x,y)=(0,b)$ , its optimal value is greater than or equal to 0.
• Linear Programming $(LP)\; hasa$ feasible solution if and only if$\left(L{P}^{The\; optimal\; value\; of\; a}\right)$is 0. In particular, if$(L{P}^{}$The artificial variable yi ( i = 1 , . . . , m ) y_i(i=1,...,m)in the optimal solution of${}^a$$)$$y_i(i=1,...,m)$ are all non-basic variables, then the optimal solution is also(LP) (LP)A basic feasible solution$of$$($$LP$$)$

Using the above method to find the initial basic feasible solution is called stage I, and after obtaining the initial feasible solution, finding the optimal solution to the original problem is called stage II. The entire solution process is called a two-stage method .

Example 2.1 (continued) : Using stage I to solve the initial feasible solution, the following problem can be constructed:
$$\begin{array}{rl}& \min y_1+y_2+y_3,\\ & s.t.-x_1+2x{\_}_2+x_3+y_1=4,\\ & 5x{\_}_1+x_2+x_4+y_2=20,\\ & 2x{\_}_1+2x{\_}_2-x_5+y_3=7,\\ & x\ge 0,y\ge 0\end{array}$$
Note that $x_3$and $x_4$ 可以看作是松弛变量，故人工变量 $y_1$ 和 $y_2$ 是多余的，只需引进 $y_3$ 为人工变量。故阶段Ⅰ的线性规划问题可简化为：
$$\begin{array}{rl}& \min y_3,\\ & s.t.-x_1+2x_2+x_3=4,\\ & 5x_1+x_2+x_4=20,\\ & 2x_1+2x_2-x_5+y_3=7,\\ & x\ge 0,y_3\ge 0\end{array}$$
Obviously, the initial basis variable is $x_B=(x_3,x_4,y_3{)}^T$ , the corresponding simplex table is shown in Table 2.5. Select$x_1$For example, $l=\min (\frac{20}{5},\frac{7}{2})$，故$y_3$Out of the basis, the new basis variable is $x_B=(x_1,x_3,y_4{)}^T$ , the corresponding simplex table is shown in Table 2.6. This gives the initial basic feasible solution$x=(\frac{7}{2},0,\frac{15}{2},\frac{5}{2},0{)}^{T.}$ _ Delete the corresponding columns in the simplex table 2.6 to obtain the initial simplex table of stage II.

2.3 Dual and dual simplex methods of linear programming

Consider the following linear programming problem:
( P ) max ⁡ { c T x ∣ A x ≤ b , x ∈ R + n } (\text{P})\quad \max\{c^{\text{T}} x |Ax\le b,x\in \R^n_{+}\}(P)max{ cTx∣Ax≤b,x∈R+n​}
此处 $A\in R^{m\times n}$，$A$ row has full rank, and the dual problem of § is defined as:
( D ) min ⁡ { b T u ∣ AT u ≥ c , u ∈ R + m } (\text{D})\quad \min\{b^{\ text{T}} u|A^{\text{T}}u\ge c,u\in \R^m_{+}\}(D)min{ bTu∣ATu≥c,u∈R+m​}
Here$R_{+}^m$means $A\; collection\; of\; m-$ dimensional nonnegative vectors. Easy to verify, the standard form of linear programming
(LP) min ⁡ { c T x ∣ A x = b , x ∈ R + n } (\text{LP})\quad \min\{c^{\text{T} } x|Ax= b,x\in \R^n_{+}\}(LP)min{ cTx∣Ax=b,x∈R+n​}
的对偶问题为：
( LD ) max ⁡ { b T u ∣ A T u ≤ c , u ∈ R + m } (\text{LD})\quad \max\{b^{\text{T}} u|A^{\text{T}}u\le c,u\in \R^m_{+}\} (LD)max{ bTu∣ATu≤c,u∈R+m​}
定理2.9 (弱对偶性) 设 $x$ 是 (LP) 的可行解，$u$ 是 (LD) 的可行解，则 $c^{\text{T}}x\ge b^{\text{T}}u$

推论2.1 若$v(LP)=-\infty$，则(LD)不可行；反之若$v(LD)=+\infty$，则(LP)不可行。即当原始线性规划问题(LP)是无界的，对偶问题(LD)一定不可行。

推论2.2 （互补松弛条件）设 $x$ 和 $u$ 分别是(LP)和(LD)的可行解，令 $s=c-A^{\text{T}}u$，则 $x$ 和 u u The necessary and sufficient conditions for$u$$s_i{x}_i=0,i=1,...,n$。

Theorem 2.10 (Strong Duality) Suppose (LP) or (LD) is feasible and the optimal solution is finite, then $v(LP)=v(LD)$。

That is, if there is at least one feasible solution in the primitive problem (LP) and the dual problem (LD), and both problems have finite optimal solutions, then their optimal objective function values ​​are equal

Corollary 2.3 For linear programming (LP) and its dual problem (LD), only one of the following four situations occurs:

1. Both (LP) and (LD) have finite optimal solutions and $v\left(LP\right)=v(LD)$;
2. $v\left(LP\right)=-\infty$ ,(LD) is not feasible;
3. $v(LD)=+\infty$,(LP)不可行；
4. (LP)和(LD)皆不可行.

推论2.4 (Farkas引理) 设$A\in {\mathbb{R}}^{m\times n},c\in {\mathbb{R}}^n$，则下列系统有且仅有一个有解：

1. $Ax\le 0,c^{T}x>0$
2. $A^{T}u=c,u\ge 0$

性质2.2 设 $A=(B,N)$，$B$ 是(LP)的对偶可行基，即 $r_N=c_N^T-c_B^T{B}^{-1}N\ge 0$ , let$BsumN$ \$N$ means BBrespectively$B$ andNNThe column index set of$N$${\overline{a}}_j=B^{-1}{a}_j,j\in \mathcal{N},\overline{b}=B^{-1}b$, let${\overline{b}}_s<0$

• 若 ${\overline{a}}_{sj}\ge 0,\forall j\in N$ , then (LP) is not feasible
• If there exists $j\in \mathcal{N}$ 使 ${\overline{a}}_{sj}<0$ , then there isBBThe adjacency dual feasible basis$B$$\overline{B}$, its column index set B ‾ = B ∪ { t } ∪ { s } \overline{\mathcal{B}}=\mathcal{B}\cup\{t\}\cup\{s\}B=B∪{ t}∪{ s } , here

t = arg ⁡ min ⁡ j ∈ N { − r j a ‾ s j ∣ a ‾ s j < 0 } (2.14) t=\arg\underset{j\in \mathcal{N}}{\min}\{ -\cfrac{r_j}{\overline{a}_{sj}}|\overline{a}_{sj}<0\}\tag{2.14} t=argj∈Nmin​{ −asj​rj​​∣asj​<0}(2.14)

Dual simplex algorithm

• Step 0 Calculate the initial dual feasible basis $B$，$A=(B,N)$
• Step 1 If $\overline{b}=B^{-1}b\ge 0$，则 $x^T=(x_B^T,0)=({\overline{b}}^T,0)\ge 0$ is the optimal solution to the original problem, otherwise, go to step 2
• Step 2 Select $s$ satisfies${\overline{b}}_s<0$，若 ${\overline{a}}_{sj}\ge 0,\forall j\in N$ , then the original problem is infeasible, otherwise$t$，令 B : = B ∪ { t } ∪ { s } {\mathcal{B}}:=\mathcal{B}\cup\{t\}\cup\{s\} B:=B∪{ t}∪{ s } , go to step 1

Example 2.1 (continued) The linear programming can be written as
$$\begin{array}{rl}& \min -7x{\_}_1-2x{\_}_2\\ & s.t.-x_1+2x{\_}_2+x_3=4,\\ & 5x{\_}_1+x_2+x_4=20,\\ & 2x{\_}_1+2x{\_}_2-x_5=7,\\ & x\ge 0\end{array}$$
Solve using the dual simplex method:

The 1st iteration. Select the initial feasible basis as $B=(a_2,a_3,a_5)$ . The corresponding simplex table is Table 2.7. Because$r_N=r_N=c_N^T-c_B^T{B}^{-1}N=(3,2)\ge 0$ , so$B=(a_2,a_3,a_5)$ is a dual feasible basis. And$x_B=\overline{b}=(-36,20,33{)}^T$ , so$B$ is not a primitive feasible base.

For the second iteration, select ${\overline{b}}_3=-36<0$，计算 $l=\min (\frac{3}{11},\frac{3}{2})=\frac{3}{11}$, so $x_1$Enter base, $x_3$Leaving the basis, the new dual feasible basis is $B=(a_1,a_2,a_5)$ , the corresponding simplex table is as follows. At this time, the feasible dual solution is$x=(\frac{36}{11},\frac{40}{11},0,0,\frac{75}{11})$ is also the original feasible solution, and thus the original optimal solution.

references

1. Integer Programming Sun Xiaoling, Li Rui Beijing, Science Press 2010
2. Convex collection
3. Operations Research S01E02—Simplex Method
4. A simple understanding of the simplex algorithm for linear programming