估计理论(2)：多元高斯变量的条件概率密度函数(PDF)

本节内容摘自Steven M. Kay，《Fundamentals of Statistical Signal Processing: Estimation Theory》。

【定理10.2】多元高斯向量的条件PDF

如果 ${\bf x}\in \mathbb{R}^{k\times 1}$ 和 ${\bf y}\in \mathbb{R}^{l\times 1}$ 为联合高斯分布随机向量，均值向量为 $[{\rm E}({\bf x})\ {\rm E}({\bf x})]^{\rm T}$ ，分块协方差矩阵为
$\tag{10.23} {\bf C}= \left[ \begin{matrix} {\bf C}_{xx} & {\bf C}_{xy} \\{\bf C}_{yx}& {\bf C}_{yy}\end{matrix} \right],$ 其中 ${\bf C}_{xx}\in \mathbb{R}^{k\times k}$ 与 ${\bf C}_{yy}\in \mathbb{R}^{l\times l}$ 分别为向量 $\bf x$ 和 $\bf y$ 的自协方差阵， ${\bf C}_{xy}\in \mathbb{R}^{k\times l}$ 与 ${\bf C}_{yx}\in \mathbb{R}^{l\times k}$ 为向量 $\bf x$ 和 $\bf y$ 的互协方差阵。显然，我们可以得到 $\bf x$ 和 $\bf y$ 的联合概率密度函数为
$p({\bf x},{\bf y})=\frac{1}{(2\pi)^{\frac{k+l}{2}}{\rm det}^{\frac{1}{2}}(\bf C)}{\rm exp}\left[-\frac{1}{2}\left( \left[\begin{matrix}{\bf x}-{\rm E}({\bf x} )\\ {\bf y}-{\rm E}({\bf y})\end{matrix}\right]\right)^{\rm T}{\bf C}^{-1}\left( \left[\begin{matrix}{\bf x}-{\rm E}({\bf x} )\\ {\bf y}-{\rm E}({\bf y})\end{matrix}\right]\right)\right],$ 因此，条件PDF $p(\bf y|x)$ 也为高斯的，且
$\tag{10.24} {\rm E}({\bf y|x})={\rm E}({\bf y})+{\bf C}_{yx}{\bf C}^{-1}_{xx}(x-{\rm E}({\bf x}))$ $\tag{10.25} {\bf C}_{y|x}={\bf C}_{yy}-{\bf C}_{yx}{\bf C}^{-1}_{xx}{\bf C}_{xy}.$ 注意，条件PDF的协方差矩阵并不依赖于 $\bf x$ ，尽管这个属性通常并不成立。

【附录10A】条件高斯PDF的推导

我们来推导【定理10.2】的结论。显然，我们有
$\begin{aligned} p({\bf y}|{\bf x})&=\frac{p({\bf x,y})}{p({\bf x})}\\ &=\frac{\frac{1}{(2\pi)^{\frac{k+l}{2}}{\rm det}^{\frac{1}{2}}(\bf C)}{\rm exp}\left[-\frac{1}{2}\left( \left[\begin{matrix}{\bf x}-{\rm E}({\bf x} )\\ {\bf y}-{\rm E}({\bf y})\end{matrix}\right]\right)^{\rm T}{\bf C}^{-1}\left( \left[\begin{matrix}{\bf x}-{\rm E}({\bf x} )\\ {\bf y}-{\rm E}({\bf y})\end{matrix}\right]\right)\right]}{\frac{1}{(2\pi)^{\frac{k}{2}}\det^{\frac{1}{2}}({\bf C}_{xx})}\exp[-\frac{1}{2}({\bf x}-{\rm E}({\bf x}))^{\rm T}{\bf C}_{xx}^{-1}({\bf x}-{\rm E}({\bf x}))]}. \end{aligned}$ 令
${\bf x}=\left[ \begin{matrix}{\bf I} & {\bf 0}\\{\bf 0}&{\bf 0}\end{matrix}\right]\left[ \begin{matrix}{\bf x}\\{\bf y}\end{matrix}\right].$ 下面我们来看分块协方差阵。由于
$\det\left(\left[ \begin{matrix} {\bf A}_{11} & {\bf A}_{12}\\ {\bf A}_{21} & {\bf A}_{22} \end{matrix} \right]\right)=\det({\bf A}_{11})\det({\bf A}_{22}-{\bf A}_{21}{\bf A}_{11} ^{-1}{\bf A}_{12} ) ,$ 可以得到
$\det {\bf C} =\det({\bf C}_{xx})\det({\bf C}_{yy}-{\bf C}_{yx}{\bf C}_{xx} ^{-1}{\bf C}_{xy}),$ 因此，有
$\frac{\det {\bf C}}{\det({\bf C}_{xx})}\det({\bf C}_{yy}-{\bf C}_{yx}{\bf C}_{xx} ^{-1}{\bf C}_{xy}).$ 如果令
$Q=\left[\begin{matrix}{\bf x}-{\rm E}({\bf x} )\\ {\bf y}-{\rm E}({\bf y})\end{matrix}\right]^{\rm T}{\bf C}^{-1} \left[\begin{matrix}{\bf x}-{\rm E}({\bf x} )\\ {\bf y}-{\rm E}({\bf y})\end{matrix}\right]-({\bf x}-{\rm E}({\bf x}))^{\rm T}{\bf C}_{xx}^{-1}({\bf x}-{\rm E}({\bf x})),$ 我们可以得到
$\begin{aligned} p({\bf y}|{\bf x})=\frac{1}{ {(2\pi)^{\frac{l}{2}}\det^{\frac{1}{2}}({\bf C}_{yy}-{\bf C}_{yx}{\bf C}_{xx} ^{-1}{\bf C}_{xy})}}\exp\left(-\frac{1}{2}Q\right). \end{aligned}$ 下面我们来求 ${\bf C}^{-1}$ ，从而得到 $Q$ 。由于对称分块矩阵的逆矩阵有
$\left[ \begin{matrix} {\bf A}_{11} & {\bf A}_{12}\\ {\bf A}_{21} & {\bf A}_{22} \end{matrix} \right]^{-1}=\left[ \begin{matrix} ({\bf A}_{11}-{\bf A}_{12}{\bf A}_{22}^{-1}{\bf A}_{21})^{-1} & -{\bf A}_{11}^{-1}{\bf A}_{12}({\bf A}_{22}-{\bf A}_{21}{\bf A}_{11}^{-1}{\bf A}_{12})^{-1}\\ -({\bf A}_{22}-{\bf A}_{21}{\bf A}_{11}^{-1}{\bf A}_{12})^{-1}{\bf A}_{21}{\bf A}_{11}^{-1} & ({\bf A}_{22}-{\bf A}_{21}{\bf A}_{11}^{-1}{\bf A}_{12})^{-1} \end{matrix} \right].$ 采用这种形式，非对角线元素互为转置，因此逆矩阵是对称的。这是由于 $\bf C$ 为对称的，因此 ${\bf C}^{-1}$ 也是对称的。根据逆矩阵性质，我们有
$({\bf A}_{11}-{\bf A}_{12}{\bf A}_{22}^{-1}{\bf A}_{21})^{-1}={\bf A}_{11}^{-1}+{\bf A}_{11}^{-1}{\bf A}_{12}({\bf A}_{22}-{\bf A}_{21}{\bf A}_{11}^{-1}{\bf A}_{12})^{-1}{\bf A}_{21}{\bf A}_{11}^{-1},$ 因此得到
${\bf C}^{-1}=\left[ \begin{matrix} {\bf C}_{xx}^{-1}+{\bf C}_{xx}^{-1}{\bf C}_{xy}{\bf B}^{-1}{\bf C}_{yx}{\bf C}_{xx}^{-1} & {\bf C}_{xx}^{-1}{\bf C}_{xy}{\bf B}^{-1}\\ -{\bf B}^{-1} {\bf C}_{yx}{\bf C}_{xx}^{-1} & {\bf B}^{-1}\end{matrix}\right],$ 其中
${\bf B}={\bf C}_{yy}-{\bf C}_{yx}{\bf C}_{xx}^{-1}{\bf C}_{xy}.$ 进一步，有
${\bf C}^{-1}=\left[ \begin{matrix} {\bf I}& -{\bf C}_{xx}^{-1}{\bf C}_{xy} \\ {\bf 0} & {\bf I}\end{matrix}\right] \left[ \begin{matrix} {\bf C}_{xx}^{-1}&{\bf 0} \\ {\bf 0} & {\bf B}^{-1}\end{matrix}\right] \left[ \begin{matrix} {\bf I}& {\bf 0}\\ -{\bf C}_{yx}{\bf C}_{xx}^{-1} & {\bf I}\end{matrix}\right].$ 再令 ${\tilde {\bf x}}={\bf x}-{\rm E}({\bf x})$ ， ${\tilde {\bf y}}={\bf y}-{\rm E}({\bf y})$ ，我们可以得到
$\begin{aligned} Q&=\left[ \begin{matrix} {\tilde {\bf x}}\\{\tilde {\bf y}} \end{matrix}\right] ^{\rm T} \left[ \begin{matrix} {\bf I}& -{\bf C}_{xx}^{-1}{\bf C}_{xy} \\ {\bf 0} & {\bf I}\end{matrix}\right] \left[ \begin{matrix} {\bf C}_{xx}^{-1}&{\bf 0} \\ {\bf 0} & {\bf B}^{-1}\end{matrix}\right] \left[ \begin{matrix} {\bf I}& {\bf 0}\\ -{\bf C}_{yx}{\bf C}_{xx}^{-1} & {\bf I}\end{matrix}\right]\left[ \begin{matrix} {\tilde {\bf x}}\\{\tilde {\bf y}} \end{matrix}\right] -{\tilde {\bf x}}^{\rm T}{\bf C}_{xx}^{-1}{\tilde {\bf x}}\\ &=\left[ \begin{matrix} {\tilde {\bf x}}\\ {\tilde {\bf y}-{\bf C}_{yx}{\bf C}_{xx}^{-1}{\tilde {\bf x}}}\end{matrix}\right]^{\rm T} \left[ \begin{matrix} {\bf C}_{xx}^{-1}&{\bf 0} \\ {\bf 0} & {\bf B}^{-1}\end{matrix}\right] \left[ \begin{matrix} {\tilde {\bf x}}\\ {\tilde {\bf y}-{\bf C}_{yx}{\bf C}_{xx}^{-1}{\tilde {\bf x}}}\end{matrix}\right]^{\rm T} -{\tilde {\bf x}}^{\rm T}{\bf C}_{xx}^{-1}{\tilde {\bf x}}\\ &=({\tilde {\bf y}-{\bf C}_{yx}{\bf C}_{xx}^{-1}{\tilde {\bf x}}})^{\rm T}{\bf B}^{-1}{(\tilde {\bf y}}-{\bf C}_{yx}{\bf C}_{xx}^{-1}{\tilde {\bf x}}) \end{aligned}$ 最终，得到
$Q=[{\bf y}-({\rm E}({\bf y})+{\bf C}_{yx}{\bf C}_{xx}^{-1}({\bf x}-{\rm E}({\bf x})))]^{\rm T} [{\bf C}_{yy}-{\bf C}_{yx}{\bf C}_{xx}^{-1}{\bf C}_{xy}]^{-1} [{\bf y}-({\rm E}({\bf y})+{\bf C}_{yx}{\bf C}_{xx}^{-1}({\bf x}-{\rm E}({\bf x})))].$ 因此均值和方差分别如(10.24)和(10.25)所示。