对偶范数共轭

Young’s inequality

$$ab\le \frac{a^p}{p}+\frac{b^q}{q}$$
其中$a\ge 0,b\ge 0,p>1,q>1,\frac{1}{p}+\frac{1}{q}=1$
当且仅当$a^p=b^q$时取等

证明：
当$a=0$或$b=0$时显然成立

当$a>0,b>0$时
由Jensen不等式
$$\begin{array}{r}\ln \left(\frac{a^p}{p}+\frac{b^q}{q}\right)\ge \frac{1}{p}\ln a^p+\frac{1}{q}\ln b^q=\ln ab\end{array}$$
当且仅当$a^p=b^q$时取等

Hölder’s inequality

$$\sum _{i=1}^n\left|x_i{y}_i\right|\le \Vert \mathbf{x}{\Vert }_p\Vert \mathbf{y}{\Vert }_q$$
其中$p\ge 1,q\ge 1,\frac{1}{p}+\frac{1}{q}=1$
$\Vert \mathbf{x}{\Vert }_p={\left({\sum }_{i=1}^n{\left|x_i\right|}^p\right)}^{\frac{1}{p}}$

当且仅当$\mathbf{x}=0$或$\mathbf{y}=0$,或$\exists c1,c2>0$，使得$c_1{\left|x_i\right|}^p=c_2{\left|y_i\right|}^q$

证明：
当$\mathbf{x}=0$或$\mathbf{y}=0$时显然成立

当$\mathbf{x}\ne 0,\mathbf{y}\ne 0$时

当$p=1$时
${\sum }_{i=1}^n\left|x_i{y}_i\right|\le \left({\max }_i\left|y_i\right|\right){\sum }_{i=1}^n\left|x_i\right|=\Vert \mathbf{x}{\Vert }_1\Vert \mathbf{y}{\Vert }_{\infty }$
同理$q=1$

当$p>1$时

设$A_i=\frac{|x_i|}{\Vert \mathbf{x}{\Vert }_p},B_i=\frac{|y_i|}{\Vert \mathbf{y}{\Vert }_q}$

由Young不等式
$$A_i{B}_i=\frac{\left|x_i{y}_i\right|}{\Vert \mathbf{x}{\Vert }_p\Vert \mathbf{y}{\Vert }_q}\le \frac{1}{p}\frac{{\left|x_i\right|}^p}{\Vert \mathbf{x}{\Vert }_p^p}+\frac{1}{q}\frac{{\left|y_i\right|}^q}{\Vert \mathbf{y}{\Vert }_q^q}$$

两边求和
$$\begin{array}{rl}\sum _{i=1}^n\frac{\left|x_i{y}_i\right|}{\Vert \mathbf{x}{\Vert }_p\Vert \mathbf{y}{\Vert }_q}& \le \sum _{i=1}^n\frac{1}{p}\frac{{\left|x_i\right|}^p}{\Vert \mathbf{x}{\Vert }_p^p}+\frac{1}{q}\frac{{\left|y_i\right|}^q}{\Vert \mathbf{y}{\Vert }_q^q}\\ \sum _{i=1}^n\frac{\left|x_i{y}_i\right|}{\Vert \mathbf{x}{\Vert }_p\Vert \mathbf{y}{\Vert }_q}& \le \frac{1}{p}\frac{{\sum }_{i=1}^n{\left|x_i\right|}^p}{\Vert \mathbf{x}{\Vert }_p^p}+\frac{1}{q}\frac{{\sum }_{i=1}^n{\left|y_i\right|}^q}{\Vert \mathbf{y}{\Vert }_q^q}\\ \sum _{i=1}^n\frac{\left|x_i{y}_i\right|}{\Vert \mathbf{x}{\Vert }_p\Vert \mathbf{y}{\Vert }_q}& \le \frac{1}{p}\frac{\Vert \mathbf{x}{\Vert }_p^p}{\Vert \mathbf{x}{\Vert }_p^p}+\frac{1}{q}\frac{\Vert \mathbf{y}{\Vert }_q^q}{\Vert \mathbf{y}{\Vert }_q^q}\\ \sum _{i=1}^n\frac{\left|x_i{y}_i\right|}{\Vert \mathbf{x}{\Vert }_p\Vert \mathbf{y}{\Vert }_q}& \le \frac{1}{p}+\frac{1}{q}\\ \sum _{i=1}^n\left|x_i{y}_i\right|& \le \Vert \mathbf{x}{\Vert }_p\Vert \mathbf{y}{\Vert }_q\end{array}$$

$$\begin{array}{rl}{c}_1{\left|x_i\right|}^p& =c_2{\left|y_i\right|}^q\\ c_1\sum _{i=1}^n{\left|x_i\right|}^p& =c_2\sum _{i=1}^n{\left|y_i\right|}^q\\ \frac{c_1}{c_2}& =\frac{\Vert \mathbf{y}{\Vert }_q^q}{\Vert \mathbf{x}{\Vert }_p^p}\end{array}$$

$$\begin{array}{rl}{c}_1{\left|x_i\right|}^p& =c_2{\left|y_i\right|}^q\\ \left|y_i\right|& =\sqrt[q]{\frac{c_1}{c_2}{\left|x_i\right|}^p}\\ \left|y_i\right|& =\frac{\Vert \mathbf{y}{\Vert }_q}{\Vert \mathbf{x}{\Vert }_p^{p-1}}{\left|x_i\right|}^{p-1}\\ \left|x_i{y}_i\right|& =\frac{\Vert \mathbf{y}{\Vert }_q}{\Vert \mathbf{x}{\Vert }_p^{p-1}}{\left|x_i\right|}^p\\ \sum _{i=1}^n\left|x_i{y}_i\right|& =\sum _{i=1}^n\frac{\Vert \mathbf{y}{\Vert }_q}{\Vert \mathbf{x}{\Vert }_p^{p-1}}{\left|x_i\right|}^p\\ \sum _{i=1}^n\left|x_i{y}_i\right|& =\Vert \mathbf{x}{\Vert }_p\Vert \mathbf{y}{\Vert }_q\end{array}$$

dual norm of p norm

设$\Vert \cdot \Vert$是定义在${\mathbb{R}}^n$的范数
对偶范数$\Vert \cdot {\Vert }_{\ast }$定义为
$$\Vert \mathbf{z}{\Vert }_{\ast }=\underset{\Vert x\Vert \le 1}{\sup }{\mathbf{z}}^T\mathbf{x}$$

也可以推出${\mathbf{x}}^T\mathbf{z}\le \Vert \mathbf{x}\Vert \Vert \mathbf{z}{\Vert }_{\ast }$

p norm

$$\Vert \mathbf{z}{\Vert }_q=\underset{\Vert x{\Vert }_p\le 1}{\sup }{\mathbf{z}}^T\mathbf{x}$$
其中$p\ge 1,q\ge 1,\frac{1}{p}+\frac{1}{q}=1$

证明：

当$\mathbf{z}=0$时，显然成立

当$\mathbf{z}\ne 0$时
由Holder不等式
$$\begin{array}{r}{\mathbf{z}}^T\mathbf{x}\le \sum _{i=1}^n\left|z_i\right|\left|x_i\right|\le \Vert \mathbf{x}{\Vert }_p\Vert \mathbf{z}{\Vert }_q\le \Vert \mathbf{z}{\Vert }_q\end{array}$$

下面验证一下取等
令$x_i=\frac{\mathrm{sign}\left(z_i\right){|z_i|}^{q-1}}{\Vert \mathbf{z}{\Vert }_q^{\frac{q}{p}}}$
$$\begin{array}{rl}\Vert \mathbf{x}{\Vert }_p^p& =\sum _{i=1}^n{\left|\frac{\mathrm{sign}\left(z_i\right){\left|z_i\right|}^{q-1}}{\Vert \mathbf{z}{\Vert }_q^{\frac{q}{p}}}\right|}^p\\ & =\sum _{i=1}^n\frac{{\left|z_i\right|}^{pq-p}}{\Vert \mathbf{z}{\Vert }_q^q}\\ & =\sum _{i=1}^n\frac{{\left|z_i\right|}^q}{\Vert \mathbf{z}{\Vert }_q^q}\\ & =\frac{\Vert \mathbf{z}{\Vert }_q^q}{\Vert \mathbf{z}{\Vert }_q^q}\\ & =1\end{array}$$
于是
$\Vert \mathbf{x}{\Vert }_p\le 1$
因此
$$\begin{array}{rl}{\mathbf{z}}^T\mathbf{x}& =\sum _{i=1}^n\frac{\mathrm{sign}\left(z_i\right){\left|z_i\right|}^{q-1}}{\Vert \mathbf{z}{\Vert }_q^{\frac{q}{p}}}{z}_i\\ & =\sum _{i=1}^n\frac{{\left|z_i\right|}^q}{\Vert \mathbf{z}{\Vert }_q^{\frac{q}{p}}}\\ & =\frac{\Vert \mathbf{z}{\Vert }_q^q}{\Vert \mathbf{z}{\Vert }_q^{\frac{q}{p}}}\\ & =\Vert \mathbf{z}{\Vert }_q\end{array}$$

conjugate

$f:{\mathbb{R}}^n\to \mathbb{R}$
共轭函数
$$f^{\ast }\left(\mathbf{y}\right)=\underset{\mathbf{x}}{\max }{\mathbf{y}}^T\mathbf{x}-f\left(\mathbf{x}\right)$$

norm

$f\left(\mathbf{x}\right)=\Vert \mathbf{x}\Vert$
$$f^{\ast }\left(\mathbf{y}\right)=\{\begin{array}{ll}0,& \Vert \mathbf{y}{\Vert }_{\ast }\le 1\\ +\infty ,& \Vert \mathbf{y}{\Vert }_{\ast }>1\end{array}$$

其中$\Vert \cdot {\Vert }_{\ast }$为对偶范数

证明：
当$\Vert \mathbf{y}{\Vert }_{\ast }\le 1$时

$${\mathbf{y}}^T\mathbf{x}\le \Vert \mathbf{x}\Vert \Vert \mathbf{y}{\Vert }_{\ast }\le \Vert \mathbf{x}\Vert$$
取$\mathbf{x}=0$，取等
$$f^{\ast }\left(\mathbf{y}\right)={\mathbf{y}}^T\mathbf{x}-\Vert \mathbf{x}\Vert =0$$

当$\Vert \mathbf{y}{\Vert }_{\ast }>1$时
由对偶范数定义
$\Vert \mathbf{y}{\Vert }_{\ast }={\max }_{\Vert \mathbf{x}\Vert \le 1}{\mathbf{x}}^T\mathbf{y}>1$
即存在$\mathbf{x}$，满足$\Vert \mathbf{x}\Vert \le 1,{\mathbf{x}}^T\mathbf{y}>1$

$$\begin{array}{rl}{f}^{\ast }\left(\mathbf{y}\right)& =\underset{\mathbf{x}}{\max }{\mathbf{y}}^T\mathbf{x}-\Vert \mathbf{x}\Vert \\ & \ge {\mathbf{y}}^T\left(t\mathbf{x}\right)-t\Vert \mathbf{x}\Vert \\ & =t\left({\mathbf{y}}^T\mathbf{x}-\Vert \mathbf{x}\Vert \right)\end{array}$$
(第二行是因为$\mathbf{x}$有任意性，所以代入$t\mathbf{x}$也是会成立的)

当$t\to \infty$时,$f^{\ast }\left(\mathbf{y}\right)\to \infty$