introduction,
Zhou Zhihua teacher recently read "machine learning" in the principal component analysis and calculate the sample covariance matrix of dimension reduction often appear during learning, and discrimination here to view this part of knowledge, in order to learn later stages of understanding.
Samples with random variables
Get sample collection process can be seen as random variables. We will distinguish between the two as far as possible to enlarge:
Random variables: At this point we already know the distribution of variables, which assumes that know the nature of system. We can calculate the variance, covariance and covariance matrix by expectations .
Sample: But it did not, the data is not the most scientific research institute random variables obtained - we do not know in advance the distribution of variables (otherwise also studied what ??), so the only information collected through the sample to estimate the unknown nature of system. Thus, the sample covariance (sample covariance) is more common .
The mathematical statistics textbook definitions: X- . 1 , X- 2 , X- . 3 , ......, X- n- independent and identically distributed and are generally X- , called X- . 1 , X- 2 , X- . 3 , ......, X- n- is from simple random sample of the overall X, referred to the sample . n is the sample size. (As to why you need an independent mutual convenience can be understood as defined in this way in subsequent applications, such as maximum likelihood estimation, easy to use).
Sample X- . 1 , X- 2 , X- . 3 , ......, X- n- figures wherein:
(1) Sample mean
(2) the sample variance
(3) Sample standard deviation
Usually, we estimate the mean and variance of random variables based on the sample mean and sample variance:
If X has the general mathematical expectation E (x) = μ, then
If X has overall variance D (X), then
Covariance,
Referring to the covariance, we usually say is in two parts: covariance (1) random variables. With mathematical expectation, the same variance, is a general parameter distribution. Covariance (2) of the sample. Is a statistic sample set can be used as a joint distribution estimate population parameters. Calculated in practice is usually a sample covariance.
To quote coolie simple-minded bloggers blog https://www.cnblogs.com/terencezhou/p/6235974.html
In the above blog, for random variables, covariance sample covariance matrix of four parts have a more comprehensive explanation. Specific explanation we can go to the above link.