Covariance matrix and correlation coefficient matrix

foreword

  This blog mainly introduces the relevant knowledge of variance, covariance and correlation coefficient, and then introduces the covariance matrix and correlation coefficient matrix, and explains it with related examples.

1. Variance, covariance and correlation coefficient

  In "Probability Theory and Mathematical Statistics", variance is used to measure a single random variable $The\; degree\; of\; dispersion\; of\; X$ , denoted as$DX$ , the calculation formula is as follows:
$$\begin{array}{rl}DX& =E(X-EX{)}^2\\ & =E{X}^2-E^{2X}\end{array}$$  The mathematical expression is: $$p^2(x)=\frac{1}{n-1}\sum _{i=1}^N(x_i-\bar{x}{)}^2$$

  Right now方差 = 平方的期望 - 期望的平方

  Covariance is used to measure two random variables $X$和$The\; degree\; of\; similarity\; between\; Y$ , denoted as$Cov(X,Y)$ ,Define as follows:
$$\begin{array}{rl}Cov(X,Y)& =And[(X-EX)\cdot (Y-EY)]\\ & =I\left(XY\right)\_-EX\cdot EY\end{array}$$  Mathematical expression: $$\sigma (x,y)=\frac{1}{n-1}\sum _{i=1}^N(x_i-\bar{x})({and}_i-\bar{y})$$

  From the formula point of view, the covariance is the difference between two variables and their own expectations, and then multiplies them, and then takes the expectation of the product. That is to say, when the value of one of the variables is greater than its own expectation, and the value of the other variable is also greater than its own expectation, that is, the trend of change of the two variables is the same, at this time, the covariance between the two variables takes a positive value . Conversely, that is, when one of the variables is greater than its own expectation, and the other variable is less than its own expectation, then the covariance between these two variables takes a negative value.

  Correlation coefficient, also called Pearson (Pearson)correlation coefficient, is used to measure two random variables $X$和$The\; degree\; of\; correlation\; between\; Y$ , recorded as$r_{XY}$, the calculation formula is:
$$r_{XY}=\frac{Cov(X,Y)}{\sqrt{DX}\sqrt{DY}}$$  If $r_{XY}>0$ , means the random variable$X$和$Y$ is positively correlated;
  if$r_{XY}<0$ , means the random variable$X$和$Y$ is negatively correlated;
  if$r_{XY}=0$ , means the random variable$X$和$Y$ is uncorrelated, that is, independent of each other;
  if$r_{XY}=\pm 1$ , means random variable$X$和$Y$ is linearly related;

  The correlation coefficient can also be regarded as a covariance: a special covariance that eliminates the impact of the dimension of the two variables and is standardized. It eliminates the influence of the range of change of the two variables, but simply reflects the change of the two variables per unit. similarity.

2. Covariance matrix

  In actual scenarios, when we describe an object, we do not describe it from only one or two dimensions. For example, when describing the performance of a neural network model, we need to consider the size, accuracy, and inference time of the model, etc. dimension to measure. When performing multidimensional data analysis, the degree of correlation between different dimensions needs to (covariance matrix)be described by a covariance matrix. The degree of correlation between dimensions constitutes a covariance matrix, and the elements on the main diagonal of the covariance matrix are The variance of the data along each dimension.
  The expression for the covariance matrix is: $$\sum =\left[\begin{array}{ccc}s(x_1,x_1)& \dots & s(x_1,x_n)\\ ⋮& \ddots & ⋮\\ s(x_n,x_1)& \dots & s(x_n,x_n)\end{array}\right]$$

3. Correlation coefficient matrix

  As the name implies, it is a matrix composed of correlation coefficients (correlation matrix), also called a coefficient matrix, and the value range of each element in the matrix is [-1, 1]​​.
  The expression of the correlation coefficient matrix is: $$\begin{array}{rl}C& =\left[\begin{array}{ccc}p(x_1,x_1)& \dots & p(x_1,x_n)\\ ⋮& \ddots & ⋮\\ p(x_n,x_1)& \dots & p(x_n,x_n)\end{array}\right]\\ & =\left[\begin{array}{ccc}1& \dots & p(x_1,x_n)\\ ⋮& \ddots & ⋮\\ p(x_n,x_1)& \dots & 1\end{array}\right]\end{array}$$