1.Lagrange function
2.Lagrange dual function of the dual problem
3.Slater Theorem
A Lagrange function:
Mark on the memory section, an optimization problem for any (not necessarily a convex optimization problem): \
{the begin Equation} \} Split the begin {\ text min {} \ {0} Quad & F_ (X) \ NEWLINE \ text { subject to:} \ quad & f_ {i} (x) \ leq 0, i = 1, ..., m \ newline & h_ {i} (x) = 0, i = 1, ..., p \ end {split} \ end {equation }
We can see that the real difficulty is that the above-mentioned problems and a group of equals does not equal constraints. The so-called "Lagrangian duality," the basic idea is that by expanding the objective function, the original problem in the objective function \ (f_ {0} \) expanded to \ (f_ {0} \) and the weighting function and constraints, that is, the original objective function and the constraint function together unified consideration, in order to achieve a simplified constraints.
We can define Lagriange function:
\ [L: D \ Times \ mathbb {R & lt} ^ {m} \ Times \ mathbb {R & lt} ^ {P} \ rightarrow \ mathbb {R & lt}, \]
\ the begin {Equation} L (x, \ lambda, \ nu ) = f_ {0} (x) + \ sum_ {i = 1} ^ {m} \ lambda_ {i} f_ {i} (x) + \ sum_ {i = 1} ^ {p} \ nu_ {i} h_ {i} (x). \ end {equation}
In this case we call \ (\ lambda_ {i} \ ) corresponding to the first \ (I \) th inequality constraints \ (f_ {i} \ leq 0 \) Lagrange multiplier, called \ (\ nu_ {i} \) corresponding to the first \ (I \) equal sign constraints \ (h_ {i} = 0 \) Lagrange multiplier.
Second Lagrange dual function and dual problem:
We define the Lagrange dual function:
\[g:\mathbb{R}^{m}\times\mathbb{R}^{p}\longrightarrow \mathbb{R},\]
\begin{equation}g(\lambda,\nu)=\inf_{x\in D}L(x,\lambda,\nu)\end{equation}
Notably, both \ (F_ {I} \) , \ (H_ {I} \) if a convex function, Lagrange dual function \ (G \) will be concave function. Further, for any \ (x \ in \ mathbb { R} ^ {n} \) satisfy the constraints (1) and \ ((\ lambda, \ nu ) \ in \ mathbb {R} ^ {m} \ Times \ R & lt mathbb {P} ^ {} \) , \ (\ the lambda \ succeq 0 \)
\ begin {split} g (\ lambda, \ nu) \ geq L (x, \ lambda, \ nu) & = f_ {0} (x) + \ sum_ {i = 1} ^ {m} \ lambda_ {i } f_ {i} (x) + \ sum_ {i = 1} ^ {p} \ nu_ {i} h_ {i} (x) \ newline & \ leq f (x), \ end {split}
on the side of an equation while pulling the infimum " \ (\ inf_ {C} \) " we obtain:
\ the begin {Equation}
G (\ the lambda, \ NU) \ Leq P ^ {\ AST}
\ End {Equation}
Now consider the following optimization problem:
\ the begin {Equation} \ the begin {Split} \ max \ Quad & G (\ lambda, \ nu) \ newline \ text {subject to:} \ quad & \ lambda \ succeq 0 \ end {split} \ end {equation}
then we say that the problem is the original problem (1) the "Lagrange dual problem ", referred to as " dual problem " .
Three geometric interpretation:
In order to establish some geometric intuition, we define the collection:
\begin{equation}\mathcal{G}\triangleq\lbrace (f_{1}(x),...,f_{m}(x),h_{1}(x),...,h_{p}(x),f_{0}(x))\in \mathbb{R}^{m}\times\mathbb{R}^{p}\times\mathbb{R}\mid x\in D \rbrace\end{equation}
This time it is easy to know:
\begin{equation}p^{\ast}=\inf\lbrace t\mid (u,v,t)\in \mathcal{G}, u\preceq 0, v=0\rbrace\end{equation}
For any \ (\ the lambda \ in \ mathbb {R & lt} ^ {m} \) , \ (\ NU \ in \ mathbb {R & lt} ^ {P} \) , \ (X \ in D \) , crosspoint \ (p \ triangleq (f_ { 1} (x), ..., f_ {m} (x), h_ {1} (x), ..., h_ {p} (x), f_ {0} (X)) \) vector \ ((\ lambda, \ nu , 1) \) perpendicular hyperplane:
\ [\ the lambda \ CDOT U + \ NU \ CDOT V + tL (X, \ the lambda, \ NU) = 0 \]
the hyperplane \ (T \) intercept the axis is exactly the Lagrange function \ ((x, \ lambda, \ nu) \) values at! ! !
We easily derived from the above observations \ (g (\ lambda, \ nu) \) geometric meaning:
\ (g (\ lambda, \ nu) \) is \ ((\ lambda, \ nu , 1) \) perpendicular to the set \ (G \) minimum !! intersecting hyperplanes intercept t- !!,
(Note that the "minimum" is not rigorous argument, the fact is infimum, but for ease of understanding and so it would be wrong, after all, we are here to describe the image !!!)
As shown above, where we draw a schematic diagram of a non-equality constraints, the corresponding two-dimensional case. As shown, \ (G (\ the lambda) \) is \ (- t \) is the slope of the line in a t axis intercept. It can be observed that if the linear translation continues downwardly it will no longer intersect and G. We noted that when \ (\ lambda \ geq 0 \ ) when, \ (G (\ the lambda) <P ^ {\ AST} \) , time \ (GAP \) is strictly greater than zero, it seems to be because when Since (G \) \ right half of the non-convexity and G, that is, \ (u \ geq 0 \) the lowest point of the left half of the lower portion than the result.
In order to facilitate research, we introduce the concept of "photogenic drawing" (Epigrah) of. We define the set:
\ begin {equation} \ mathcal { A} = \ lbrace p + (u, 0, t) \ in \ mathbb {R} ^ {m} \ times \ mathbb {R} ^ {p} \ times \ mathbb {R} \ mid p \ in \ mathcal { G}, u \ in \ mathbb {R} ^ {m}, u \ succeq 0, t \ in \ mathbb {R}, t \ geq \ rbrace \ end {equation}
saying the optimization problem (1) of the mirror of FIG. (Epigrah). Readily seen, the mirror is FIG \ (\ mathcal {G} \ ) a series of forward translation constituted.
As shown, we plotted here, and below the upper case of FIG mirror FIG \ (\ mathcal {A} \ ) of FIG.
We easily verify the following properties:
Property 1: If the original problem (1) is a convex optimization problem, that is, \ (F_ {I} \) , I = 0, .., m are convex function, and \ (H_ {I} \) , I = 1, ..., p are affine function of time, on which the mirror of FIG \ (\ mathcal {a} \ ) is a convex set.
Four Slater condition:
With the above groundwork, we can introduce a result, it tells us, under what conditions convex optimization problem and it is a strong duality of Lagrange dual problem, that is, under what conditions we can convert the original problem. Fortunately, this condition tells us that, in general, a strong duality is true, because the condition is very weak.
Theorem: If the original problem (1) is a convex optimization problem, the presence \ (\ tilde {x} \ in \ text {relint} D \) such that: \ (F_ {I} (\ tilde are {X}) <0 \ ) , for any \ (I =. 1, ..., m \) , then the primal and the dual problem of a strong duality.
Proof:
We may assume that an affine function:
\ (H_ {I} (X) = \ sum_. 1} = {J} ^ {n-A_ X_ {ij of {J}} + {I} B_ \) , and the matrix \ ( a = (a_ {ij}) \) satisfies \ (Rank (a) = P \) , or we can further reduce the number of equality constraints to obtain equivalent convex optimization problem, and \ (d ^ {\ ast } \) remains unchanged.
We make collections:
\ [\ mathcal {B} = \ lbrace (U, 0, T) \ in \ mathbb {R & lt} ^ {m} \ Times \ mathbb {R & lt} ^ {P} \ Times \ mathbb {R & lt} \ U MID \ preceq 0, T <P ^ {\ AST} \ rbrace \] ,
At this time, \ (\ mathcal {B} \ ) and the current mirror \ (\ mathcal {A} \ ) intersection is empty, they are empty set. Thus a convex set separation theorem, there are two collection hyperplane separation, i.e. there \ ((\ lambda_ {0} , \ nu_ {0}, t_ {0}) \ neq 0 \ in \ mathbb {R} ^ { m} \ times \ mathbb {R } ^ {p} \ times \ mathbb {R} \) and \ (b \ in \ mathbb { R} \) such that:
for any \ (X \ in D \) , \ (\ xi \ in \ mathbb { R} _ {+} ^ {m} \) and \ (T \ in \ mathbb {R & lt} _ {+} \) : \
the begin {Equation} \ sum_ {I =. 1} ^ {m} \ lambda_ {0 , i} (f_ {i} (x) + \ xi_ {i}) + \ sum_ {i = 1} ^ {p} v_ {0, i} h_ {i} (x ) + t_ {0} (f_ {0} (x) + t) \ geq b \ end {equation}
and for any \ (U \ in \ R & lt mathbb {m} ^ {} \) ,\ (U \ preceq 0 \) , \ (T <P ^ {\ AST} \) :
\ the begin {Equation} \ lambda_ {0} \ CDOT U + T_ {0} T \ Leq B \ End {Equation}
by the optionally in (9) we can immediately know \ (\ {0} lambda_ \ succeq 0 \) , \ (T_ {0} \ GEQ 0 \) , then we make (10) \ (u \ rightarrow 0 \) , \ (T \ rightarrow P ^ {\ AST} \) , can know: \ (T_ {0} P ^ {\ AST} \ Leq B \) , so we combined (9) can be seen for any \ ( X \ in D \) : \
the begin {Equation} \ sum_ {I =. 1} ^ {m} \ lambda_ {0, I} F_ {I} (X) + \ sum_ {I =. 1} ^ {P} V_ {0, i} h_ {i } (x) + t_ {0} f_ {0} (x) \ geq t_ {0} p ^ {\ ast}. \ end {equation}
we note that, if this time \ (t_ {0}> 0 \ ) is on both sides simultaneously formula dividing \ (t_ {0} \) we immediately for any \ (X \ in D \) : \
[L (X, \ {0} lambda_ / t_ {0}, \ nu_ {0} / t_ {0}) \ geq p ^ {\ ast}, \]
Now then we get:
\ (G (\ {0} lambda_ / T_ {0}, \ {0} nu_ / T_ {0}) \ P ^ {GEQ \ AST} \) , then DUALITY.
At this point we assume \ (t_ {0}> 0 \) is not satisfied, then the \ (T_ {0} = 0 \) , for any \ (X \ in D \) : \
the begin {Equation} \ sum_ {I =. 1 } ^ {m} \ lambda_ { 0, i} f_ {i} (x) + \ sum_ {i = 1} ^ {p} v_ {0, i} h_ {i} (x) \ geq 0. \ end {equation}
this case, since \ (\ tilde are {X} \ in \ {text} relint D \) , and \ (f_ {i} (\ tilde {x}) <0 (i = 1, ..., m ) \) , there is a \ (X \) in \ (D \) affine in the closure art \ (the U-\) , \ (the U-\ Subset D \) , and \ (f_ {i} <0 (i = 1, ..., m ) \) constant established on D, case binding \ (\ lambda_ {0, i } \ geq 0 \) we now know that for any \ (X \ in the U-\) :
\ begin {equation} \ sum_ { i = 1} ^ {p} v_ {0, i} h_ {i} (x) \ geq - \ sum_ {i = 1} ^ {m} \ lambda_ {0, i} f_ {i} (x) \ geq 0 \ end {equation}
noted affine function:\ (\ sum_ {i = 1 } ^ {p} v_ {0, i} h_ {i} \) in (\ tilde {x} \) \ taken at \ (0 \) , if it is non-constant at \ ( 0 \) , is bound in \ (the U-\) values in the positive and negative, so \ (\ sum_ {i = 1 } ^ {p} v_ {0, i} h_ {i} \) identically zero, By assumption \ (rank (A) = p \) we obtain immediate \ (\ nu_ {0} = 0 \) , then:
\ the begin {Equation} \ sum_ {I =. 1} ^ {m} \ lambda_ {0, i} f_ {i} (\ tilde {x}) \ geq 0, \ end {equation}
this case, since \ (\ lambda_ {0, I} \ GEQ 0 \) , \ (F_ {I} (\ {tilde are } X) <0 \) , \ (. 1 = I, ..., m \) we obtain immediate \ (\ lambda_ {0} = 0 \) , which \ ((\ lambda_ {0} , \ nu_ { 0}, t_ {0}) \ neq 0 \) conflicts, then \ (t_ {0} \) must be greater than 0, the proposition is proved.