全部笔记的汇总贴（视频也有传送门）：中科大-凸优化
一、对任意优化问题
( P ) min f 0 ( x ) s . t . f i ( x ) ≤ 0 , i = 1 , ⋯ , m ( P ∗ ) h i ( x ) = 0 , i = 1 , ⋯ , P L ( x , λ , v ) = f 0 ( x ) + ∑ i = 1 m λ i f i ( x ) + ∑ i = 1 P v i h i ( x ) g ( λ , v ) = inf x ∈ D L ( x , λ , v ) (P)\min f_0(x)\\s.t.\;f_i(x)\le0,i=1,\cdots,m\;\;\;(P^*)\\h_i(x)=0,i=1,\cdots,P\\L(x,\lambda,v)=f_0(x)+\sum_{i=1}^m\lambda_if_i(x)+\sum_{i=1}^Pv_ih_i(x)\\g(\lambda,v)=\inf_{x\in D}L(x,\lambda,v) (P)minf0(x)s.t.fi(x)≤0,i=1,⋯,m(P∗)hi(x)=0,i=1,⋯,PL(x,λ,v)=f0(x)+i=1∑mλifi(x)+i=1∑Pvihi(x)g(λ,v)=x∈DinfL(x,λ,v)
( D ) max g ( λ , v ) s . t . λ ≥ 0 R m + P d ∗ (D)\max g(\lambda,v)\\s.t.\;\lambda\ge0\\\R^{m+P}\;\;\;\;\;d^* (D)maxg(λ,v)s.t.λ≥0Rm+Pd∗
- 对偶问题一定是凸优化问题
- d ∗ ≤ P ∗ d^*\le P^* d∗≤P∗ （weak duality，一定成立）
d ∗ = P ∗ d^*=P^* d∗=P∗ （strong duality）
P ∗ − d ∗ P^*-d^* P∗−d∗ （duality gap）
D的Relative Interior（相对内部）
Relint D= { x ∈ D ∣ ∃ r > 0 , B ( x , r ) ∩ a f f D ≤ D } \{ x\in D|\exists r>0,B(x,r)\cap aff D\le D\} {
x∈D∣∃r>0,B(x,r)∩affD≤D}
Slater’s Condition （ P ∗ = d ∗ P^*=d^* P∗=d∗的充分条件）
若有凸问题 min f 0 ( x ) s . t . f i ( x ) ≤ 0 , i = 1 , ⋯ , m , A x = b \min f_0(x)\\s.t. f_i(x)\le0,i=1,\cdots,m,Ax=b minf0(x)s.t.fi(x)≤0,i=1,⋯,m,Ax=b其中 f i ( x ) f_i(x) fi(x)为凸， ∀ i \forall i ∀i，当 ∃ x ∈ r e l i n t D \exist x\in relint D ∃x∈relintD，使 f i ( x ) ≤ 0 , i = 1 , ⋯ , m , A x = b f_i(x)\le0,i=1,\cdots,m,Ax=b fi(x)≤0,i=1,⋯,m,Ax=b满足时， P ∗ = d ∗ P^*=d^* P∗=d∗
A weaker Slater’s Condition（充分条件）
若不等式约束为仿射时，只要可行域非空，必有 P ∗ = d ∗ P^*=d^* P∗=d∗
c x + d ≤ 0 ( 半 空 间 ) c x + d = 0 ( 超 平 面 ) A x = b cx+d\le0(半空间)\;\;\;\;\;cx+d=0(超平面)\;\;\;\;\;\;\;\;Ax=b cx+d≤0(半空间)cx+d=0(超平面)Ax=b
r e l i n t D = r e l i n t { d o m f 0 ∩ ∩ i d o m f i } = r e l i n t { d o m f 0 } relint D=relint\{dom\;f_0\cap \cap_idom\;f_i\}\\=relint\{dom\;f_0\} relintD=relint{
domf0∩∩idomfi}=relint{
domf0}
线性规划问题若可行，必有 P ∗ = d ∗ P^*=d^* P∗=d∗
例：
min X T X s . t . A X = b ( P ∗ = d ∗ ) ⇔ ( D ) max v − 1 4 V T A A T V − b T V \min X^TX\\s.t. AX=b\\(P^*=d^*)\\\Leftrightarrow(D)\max_v-\frac14V^TAA^TV-b^TV minXTXs.t.AX=b(P∗=d∗)⇔(D)vmax−41VTAATV−bTV
例：QCQP
{ min 1 2 X T P 0 X + q 0 T X + r 0 s . t . 1 2 X T P i X + q i T X + r i ≤ 0 , i = 1 , ⋯ , m P 0 ∈ S + + n , P i ∈ S + n ⇒ L ( x , λ ) = 1 2 X T P 0 X + q 0 T X + r 0 + ∑ i = 1 m ( 1 2 λ i X T P i X + λ i q i T X + λ i r i ) = 1 2 X T ( P 0 + ∑ i = 1 m λ i P i ) X + ( q 0 + ∑ i = 1 m λ i q i ) T X + ( r 0 + ∑ i = 1 m λ i r i ) ⇒ g ( λ ) = inf x L ( x , λ ) ( λ ≥ 0 ) = − 1 2 Q T ( λ ) P − 1 ( λ ) Q ( λ ) + R ( λ ) { max − 1 2 Q T ( λ ) P − 1 ( λ ) Q ( λ ) + R ( λ ) s . t . λ ≥ 0 ∃ x ∈ D = R n ; 1 2 X T P i X + q i X + r i < 0 , i = 1 , ⋯ , m 则 P ∗ + d ∗ 若 q i = 0 , r i = 0 , 1 2 X T P i X < 0 , 此 时 P ∗ = d ∗ = 0 \left\{ \begin{array}{l} \min \frac12X^TP_0X+q_0^TX+r_0 \\ \\s.t. \frac12X^TP_iX+q_i^TX+r_i\le0,i=1,\cdots,m \end{array} \right.\\P_0\in S_{++}^n,P_i\in S_+^n \\\Rightarrow L(x,\lambda)=\frac12X^TP_0X+q_0^TX+r_0+\sum_{i=1}^m(\frac12\lambda_iX^TP_iX+\lambda_iq_i^TX+\lambda_ir_i)\\=\frac12X^T(P_0+\sum_{i=1}^m\lambda_iP_i)X+(q_0+\sum_{i=1}^m\lambda_iq_i)^TX+(r_0+\sum_{i=1}^m\lambda_ir_i)\\\Rightarrow g(\lambda)=\inf_x L(x,\lambda)\\(\lambda\ge0)=-\frac12Q^T(\lambda)P^{-1}(\lambda)Q(\lambda)+R(\lambda)\\\left\{ \begin{array}{l} \max -\frac12Q^T(\lambda)P^{-1}(\lambda)Q(\lambda)+R(\lambda) \\ \\s.t.\;\lambda\ge0 \end{array} \right.\\\exist x\in D=\R^n;\frac12X^TP_iX+q_iX+r_i<0,i=1,\cdots,m则P^*+d^*\\若q_i=0,r_i=0,\frac12X^TP_iX<0,此时P^*=d^*=0 ⎩⎨⎧min21XTP0X+q0TX+r0s.t.21XTPiX+qiTX+ri≤0,i=1,⋯,mP0∈S++n,Pi∈S+n⇒L(x,λ)=21XTP0X+q0TX+r0+i=1∑m(21λiXTPiX+λiqiTX+λiri)=21XT(P0+i=1∑mλiPi)X+(q0+i=1∑mλiqi)TX+(r0+i=1∑mλiri)⇒g(λ)=xinfL(x,λ)(λ≥0)=−21QT(λ)P−1(λ)Q(λ)+R(λ)⎩⎨⎧max−21QT(λ)P−1(λ)Q(λ)+R(λ)s.t.λ≥0∃x∈D=Rn;21XTPiX+qiX+ri<0,i=1,⋯,m则P∗+d∗若qi=0,ri=0,21XTPiX<0,此时P∗=d∗=0