凸集、凸函数及其充分必要条件

凸集的定义：

设集合$D \subset {R^n}$，若对于任意点$x,y \in D$及实数$\alpha \in \left[ {0,1} \right]$，都有$\alpha x + \left( {1 - \alpha } \right)y \in D$
则称集合$D$为凸集。
由凸集的定义可以看出凸集的几何意义，对于非空集合$D$，连接$D$中任意两点$x,y$的线段仍属于该集合，则该集合$D$是凸集。
图1所示的图形是凸集，图2显示的图形是非凸集。

凸函数定义：

设函数$f\left( x \right)$定义在凸集$D \subset {R^n}$上，若对于任意的$x,y \in D$及任意实数$\alpha \in \left[ {0,1} \right]$，都有$f\left[ {\alpha x + \left( {1 - \alpha } \right)y} \right] \le \alpha f\left( x \right) + \left( {1 - \alpha } \right)f\left( y \right)$，则称$f\left( x \right)$为凸集$D$上的凸函数。

凸函数的充分必要条件：

（一阶条件）

设在凸集$D \subset {R^n}$上$f\left( x \right)$可微，则$f\left( x \right)$在$D$上为凸函数的充分必要条件是对任意的$x,y \in D$都有$f\left( y \right) \ge f\left( x \right) + \nabla f{\left( x \right)^T}\left( {y - x} \right)$
证明：
必要性。 设$f\left( x \right)$是$D$上的凸函数。任取$x,y \in D$及$\alpha \in \left[ {0,1} \right]$，有

$$f\left[ {\alpha y + \left( {1 - \alpha } \right)x} \right] \le \alpha f\left( y \right) + \left( {1 - \alpha } \right)f\left( x \right)$$

即

$$f\left[ {x + \alpha \left( {y - x} \right)} \right] \le f\left( x \right) + \alpha \left[ {f\left( y \right) - f\left( x \right)} \right]$$
由泰勒公式有 $$f\left[ {x + \alpha \left( {y - x} \right)} \right] = f\left( x \right) + \alpha \nabla f{\left( x \right)^T}\left( {y - x} \right) + o\left( {\left\| {\alpha \left( {y - x} \right)} \right\|} \right)$$

代入上式得

$$f\left( y \right) - f\left( x \right) \ge \nabla f{\left( x \right)^T}\left( {y - x} \right) + \frac{{o\left( {\left\| {\alpha \left( {y - x} \right)} \right\|} \right)}}{\alpha }$$

上式两端取极限，令$\alpha \to 0$有

$$f\left( y \right) \ge f\left( x \right) + \nabla f{\left( x \right)^T}\left( {y - x} \right)$$

充分性。因为$D$为凸集，所以设任意的$x,y \in D$，$\alpha \in \left[ {0,1} \right]$，则$\alpha x + \left( {1 - \alpha } \right)y \in D$

令$\alpha x + \left( {1 - \alpha } \right)y = z$，有

$$\begin{array}{l} f\left( x \right) - f\left( z \right) \ge \nabla f{\left( z \right)^T}\left( {x - z} \right)\\ f\left( y \right) - f\left( z \right) \ge \nabla f{\left( z \right)^T}\left( {y - z} \right) \end{array}$$

用$\alpha ,1 - \alpha$分别乘上面两式得

$$\begin{array}{l} \alpha \left[ {f\left( x \right) - f\left( z \right)} \right] \ge \alpha \nabla f{\left( z \right)^T}\left( {x - z} \right)\\ \left( {1 - \alpha } \right)\left[ {f\left( y \right) - f\left( z \right)} \right] \ge \left( {1 - \alpha } \right)\nabla f{\left( z \right)^T}\left( {y - z} \right) \end{array}$$

相加并整理得

$$\begin{array}{l} \alpha \left[ {f\left( x \right) - f\left( z \right)} \right] + \left( {1 - \alpha } \right)\left[ {f\left( y \right) - f\left( z \right)} \right] \ge \alpha \nabla f{\left( z \right)^T}\left( {x - z} \right) + \left( {1 - \alpha } \right)\nabla f{\left( z \right)^T}\left( {y - z} \right)\\ \alpha f\left( x \right) - \alpha f\left( z \right) + f\left( y \right) - f\left( z \right) - \alpha f\left( y \right) + \alpha f\left( z \right) \ge \nabla f{\left( z \right)^T}\left[ {\alpha x - \alpha z + y - z - \alpha y + \alpha z} \right]\\ \alpha f\left( x \right) + \left( {1 - \alpha } \right)f\left( y \right) - f\left( z \right) \ge \nabla f{\left( z \right)^T}\left[ {\alpha x + \left( {1 - \alpha } \right)y - z} \right] = 0\\ \alpha f\left( x \right) + \left( {1 - \alpha } \right)f\left( y \right) \ge f\left( z \right) \end{array}$$
即 $\alpha f\left( x \right) + \left( {1 - \alpha } \right)f\left( y \right) \ge f\left[ {\alpha x + \left( {1 - \alpha } \right)y} \right]$
故 $f\left( x \right)$在 $D$上是凸函数。

（二阶条件）

设在开凸集$D \subset {R^n}$内$f\left( x \right)$二阶可微，则$f\left( x \right)$是$D$内的凸函数的充分必要条件为在$D$内任意一点$x$处，$f\left( x \right)$的海色(Hesse)矩阵$G\left( x \right)$半正定，其中

$$G\left( x \right) = {\nabla ^2}f\left( x \right) = \left[ \begin{array}{l} \frac{{{\partial ^2}f}}{{\partial {x_1}^2}}{\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \frac{{{\partial ^2}f}}{{\partial {x_1}\partial {x_2}}}{\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \cdots {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \frac{{{\partial ^2}f}}{{\partial {x_1}\partial {x_n}}}\\ \frac{{{\partial ^2}f}}{{\partial {x_2}\partial {x_1}}}{\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \frac{{{\partial ^2}f}}{{\partial {x_2}^2}}{\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \cdots {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \frac{{{\partial ^2}f}}{{\partial {x_2}\partial {x_n}}}\\ {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \vdots {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \vdots {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \vdots \\ \frac{{{\partial ^2}f}}{{\partial {x_n}\partial {x_1}}}{\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \frac{{{\partial ^2}f}}{{\partial {x_n}\partial {x_2}}}{\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \cdots {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \frac{{{\partial ^2}f}}{{\partial {x_n}^2}} \end{array} \right]$$

证明:
必要性。任取$x \in D$及$y \in {R^n}\left( {y \ne 0} \right)$，因为$D$为开集，所以存在$\varepsilon > 0$，当$\alpha \in \left[ { - \varepsilon ,\varepsilon } \right]$时，$x + \alpha y \in D$，由一阶条件可得

$$f\left( {x + \alpha y} \right) \ge f\left( x \right) + \alpha \nabla f{\left( x \right)^T}y$$

由泰勒公式有

$$f\left( {x + \alpha y} \right) = f\left( x \right) + \alpha \nabla f{\left( x \right)^T}y + \frac{1}{2}{\alpha ^2}{y^T}G\left( x \right)y + o\left( {{\alpha ^2}} \right)$$

由此可得

$$\frac{1}{2}{\alpha ^2}{y^T}G\left( x \right)y + o\left( {{\alpha ^2}} \right) \ge 0$$

所以

$${y^T}G\left( x \right)y + \frac{{o\left( {{\alpha ^2}} \right)}}{{{\alpha ^2}}} \ge 0$$

令$\alpha \to 0$，取极限得

$${y^T}G\left( x \right)y \ge 0$$

即$G\left( x \right)$半正定。

充分性。任取$x,y \in D$，因为$G\left( x \right)$半正定，由泰勒公式可得

$$\begin{array}{l} f\left( y \right) = f\left( x \right) + \nabla f{\left( x \right)^T}\left( {y - x} \right) + \frac{1}{2}{\left( {y - x} \right)^T}G\left( \xi \right)\left( {y - x} \right)\\ {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} {\kern 1pt} \ge f\left( x \right) + \nabla f{\left( x \right)^T}\left( {y - x} \right) \end{array}$$

其中$\xi = x + \alpha \left( {y - x} \right),\alpha \in \left( {0,1} \right)$
由一阶条件可得$f\left( x \right)$为凸函数。