【人工智能】3 主成分分析法(PCA)

参考链接:http://blog.codinglabs.org/articles/pca-tutorial.html

1. 为什么要用主成分分析法?

  主成分分析法力图在保证数据丢失最少的原则下,对多维数据进行降维,是一个化繁为简的过程。

  当一个样本的维度过高时,计算的复杂度会变得很大,而且某些维度的数据其实并没有起到什么作用,放在一起进行计算反而会使结果失真,于是就需要运用 主成分分析法(PCA)根据贡献度,将信息浓缩在几个新维度上。

2. 主成分分析法步骤

  现有 \(m\)\(n\) 维数据集,把这些原始数据组成 \(n\)\(m\) 列的矩阵 \(X\)
\[ X_{n×m}=\left[ \begin{matrix} x_{1,1}&x_{1,2}&x_{1,3}&\cdots&x_{1,m}\\ x_{2,1}&x_{2,2}&x_{2,3}&\cdots&x_{2,m}\\ x_{3,1}&x_{3,2}&x_{3,3}&\cdots&x_{3,m}\\ \vdots&\vdots&\vdots&\ddots&\vdots\\ x_{n,1}&x_{n,2}&x_{n,3}&\cdots&x_{n,m}\\ \end{matrix} \right]_{n×m} \]
  然后进行去均值化,得到 \(\hat{X}\) :
\[ \hat{X}_{n×m}= \left[ \begin{matrix} x_{1,1}-\bar{X}_{1}&x_{1,2}-\bar{X}_{1}&x_{1,3}-\bar{X}_{1}&\cdots&x_{1,m}-\bar{X}_{1}\\ x_{2,1}-\bar{X}_{2}&x_{2,2}-\bar{X}_{2}&x_{2,3}-\bar{X}_{2}&\cdots&x_{2,m}-\bar{X}_{2}\\ x_{3,1}-\bar{X}_{3}&x_{3,2}-\bar{X}_{3}&x_{3,3}-\bar{X}_{3}&\cdots&x_{3,m}-\bar{X}_{3}\\ \vdots&\vdots&\vdots&\ddots&\vdots\\ x_{n,1}-\bar{X}_{n}&x_{n,2}-\bar{X}_{n}&x_{n,3}-\bar{X}_{n}&\cdots&x_{n,m}-\bar{X}_{n}\\ \end{matrix} \right]_{n×m} \\ \bar{X}_{i}=\frac{\sum_{k=1}^m x_{i,k}}{m} \]
  接下来求 协方差矩阵 \(C​\):
\[ C_{n×n}=\frac{1}{m}\hat{X}_{n×m}\hat{X}^T_{m×n} \]
  计算协方差矩阵 \(C\)特征值 \(\lambda\)特征向量 \(c\),这里得到了 \(n\) 个新维度:

\[ \lambda=\left[ \begin{matrix} \lambda_1& \lambda_2& \lambda_3 &\cdots& \lambda_n \end{matrix} \right]\\ \]

\[ c= \begin{cases} c_1=\left[ \begin{matrix} a_{1,1}&a_{1,2}&a_{1,3}&\cdots&a_{1,n} \end{matrix} \right]\\ c_2=\left[ \begin{matrix} a_{2,1}&a_{2,2}&a_{2,3}&\cdots&a_{2,n} \end{matrix} \right]\\ c_3=\left[ \begin{matrix} a_{3,1}&a_{3,2}&a_{3,3}&\cdots&a_{3,n} \end{matrix} \right]\\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \vdots \\ c_n=\left[ \begin{matrix} a_{n,1}&a_{n,2}&a_{n,3}&\cdots&a_{n,n} \end{matrix} \right]\\ \end{cases} \]

  接下来计算每个新维度的 贡献率 \(\eta​\)
\[ \eta= \left[ \begin{matrix} \eta_1& \eta_2& \eta_3 &\cdots& \eta_n \end{matrix} \right]\\ \eta_i=\frac{\lambda_i}{\sum_{k=1}^n \lambda_{k}}×100 \% \]
  按照从高到低的次序对 贡献率 \(\eta\) 进行排序,得到每个新维度排序后的 特征值 \(\hat{\lambda}\)特征向量 \(\hat{c}\)
$$
特征值:\
\hat{\lambda}=\left[
\begin{matrix}
\hat{\lambda_1}& \hat{\lambda_2}& \hat{\lambda_3} &\cdots& \hat{\lambda_n}
\end{matrix}
\right]\

特征向量:\
\hat{c}=
\begin{cases}
\hat{c_1}=\left[
\begin{matrix}
\hat{a_{1,1}}&\hat{a_{1,2}}&\hat{a_{1,3}}&\cdots&\hat{a_{1,n}}
\end{matrix}
\right]\
\hat{c_2}=\left[
\begin{matrix}
\hat{a_{2,1}}&\hat{a_{2,2}}&\hat{a_{2,3}}&\cdots&\hat{a_{2,n}}
\end{matrix}
\right]\
\hat{c_3}=\left[
\begin{matrix}
\hat{a_{3,1}}&\hat{a_{3,2}}&\hat{a_{3,3}}&\cdots&\hat{a_{3,n}}
\end{matrix}
\right]\
                              \vdots
\
\hat{c_n}=\left[
\begin{matrix}
\hat{a_{n,1}}&\hat{a_{n,2}}&\hat{a_{n,3}}&\cdots&\hat{a_{n,n}}
\end{matrix}
\right]\

\end{cases}
\[   从高到低计算累计贡献率,直到累计贡献率达到理想值(一般为90%)时,取此时到之前的维度为新维度,假累计到第 $h$ 维时,累计贡献率到达理想值,从 **$n$ 维降到 $h$ 维的新数据集 $Y$** 的计算方法为**: \]
Y_{h×m}=\left[
\begin{matrix}
y_{1,1}&y_{1,2}&y_{1,3}&\cdots&y_{1,m}\
y_{2,1}&y_{2,2}&y_{2,3}&\cdots&y_{2,m}\
y_{3,1}&y_{3,2}&y_{3,3}&\cdots&y_{3,m}\
\vdots&\vdots&\vdots&\ddots&\vdots\
y_{h,1}&y_{h,2}&y_{h,3}&\cdots&y_{h,m}\
\end{matrix}
\right]{h×m}
\
y
{i,j}=x_{1,j}×\hat{a_{i,1}}+x_{2,j}×\hat{a_{i,2}}+...+x_{n,j}×\hat{a_{i,n}}\
换言之:\
Y_{h×m}=
\left[
\begin{matrix}
\hat{a_{1,1}}&\hat{a_{1,2}}&\hat{a_{1,3}}&\cdots&\hat{a_{1,n}}\
\hat{a_{2,1}}&\hat{a_{2,2}}&\hat{a_{2,3}}&\cdots&\hat{a_{2,n}}\
\hat{a_{3,1}}&\hat{a_{3,2}}&\hat{a_{3,3}}&\cdots&\hat{a_{3,n}}\
\vdots&\vdots&\vdots&\ddots&\vdots\
\hat{a_{h,1}}&\hat{a_{h,2}}&\hat{a_{h,3}}&\cdots&\hat{a_{h,n}}\
\end{matrix}
\right]{h×n}
×
\left[
\begin{matrix}
x_{1,1}&x_{1,2}&x_{1,3}&\cdots&x_{1,m}\
x_{2,1}&x_{2,2}&x_{2,3}&\cdots&x_{2,m}\
x_{3,1}&x_{3,2}&x_{3,3}&\cdots&x_{3,m}\
\vdots&\vdots&\vdots&\ddots&\vdots\
x_{n,1}&x_{n,2}&x_{n,3}&\cdots&x_{n,m}\
\end{matrix}
\right]
{n×m}
$$

猜你喜欢

转载自www.cnblogs.com/yznnnn/p/10548686.html
0条评论
添加一条新回复