Statistics_Jia Junping——Thinking Questions Chapter 11 Univariate Linear Regression

1. Explain the meaning of correlation and explain the characteristics of correlation.

The uncertain quantitative relationship between variables is called correlation.

The characteristics of the correlation relationship: the value of one variable cannot be uniquely determined by another variable, when the variable xxWhen$x$ takes a certain value, the variable yyThere may be several values ​​for$y\; .$Variables for which this relationship is uncertain cannot be described by a functional relationship.

2. What problems does the correlation analysis mainly solve?

Correlation analysis is the description and measurement of the linear relationship between two variables, and the problems it needs to solve include:

(1) Whether there is a relationship between the variables;
(2) If there is a relationship, what kind of relationship between them;
(3) How strong is the relationship between the variables;
(4) The relationship between the variables reflected by the sample can be No represents the relationship between population variables.

3. What are the basic assumptions in the correlation analysis?

When conducting correlation analysis, there are two main assumptions about the population:

(1) There is a linear relationship between the two variables;
(2) Both variables are random variables.

4. Briefly describe the nature of the correlation coefficient.

The correlation coefficient is a statistic that measures the strength of the linear relationship between two variables, calculated from sample data. If the correlation coefficient is calculated based on all the overall data, it is called the overall correlation coefficient, denoted as ρ; if it is calculated based on the sample data, it is called the sample correlation coefficient, denoted as $r$。

Properties of the correlation coefficient:

（1）$The\; value\; range\; of\; r$ is$-1\sim +1$ , ie$-1\le r\le 1$。若 $0<r\le 1$ , indicating$x$与yyThere is a positive linear correlation between$y$$-1\le r<0$ , indicating that$x$与yyThere is a negative linear correlation between$y$$r=+1$ , indicating$x$与yyThere is a complete positive linear correlation between$y$$r=-1$ , indicating$x$与yyThere is a complete negative linear correlation between$y\; .$It can be seen that when$|r|=1$时，$The\; value\; of\; y$ depends entirely on$x$ , the relationship between the two is a function; when$r=0$ , indicating$The\; value\; of\; y$ and$x$ is independent, that is, there is no linear correlation between the two.
(2)$r$ has symmetry. $x$与yyCorrelation coefficient rxybetween$y\; r\_\{xy\}$$r_{xy}$和$y$ andxxThe correlation coefficient ryx between$x\; is\; equal,\; ie\; rxy=ryx.$
(3)$r$ value size and$x$和$The\; origin\; of\; y$ is independent of scale. change$x$和yyThe data origin and measurement scale of$y$$The\; value\; of\; r$ .
(4)$r$ is just$x$与yyA measure of the linear relationship between$y\; ,\; it\; cannot\; be\; used\; to\; describe\; nonlinear\; relationships.$This means,$r=0$ only means that there is no linear correlation between the two variables, it does not mean that there is no relationship between the variables, and there may be a nonlinear correlation between them. When the nonlinear correlation between variables is large, it may lead to$r=0$ . Therefore, when$r=When\; it\; is\; 0$ or very small, it cannot be easily concluded that there is no correlation between the two variables, but a reasonable explanation should be made in combination with the scatter plot.
(5)$Although\; r$ is a measure of the linear relationship between two variables, it does not necessarily mean that$x$与$y$ must have a causal relationship.

5. Why do we need to test the significance of the correlation coefficient?

In general, the overall correlation coefficient ρ is unknown, usually based on the sample correlation coefficient $r$ as an approximate estimate of ρ. but since$r$ is calculated from sample data and it is affected by sampling fluctuations. Due to the different samples drawn,$The\; value\; of\; r$ is also different, so$r$ is a random variable. Whether the degree of overall correlation can be explained based on the sample correlation coefficient needs to examine the reliability of the sample correlation coefficient, that is, to conduct a significance test.

6. Briefly describe the steps of the significance test of the correlation coefficient.

The steps to test the significance of the correlation coefficient are as follows:

(1) Propose a hypothesis, H0: ρ=0; H1: ρ≠0.
(2) Calculate the statistics of the test,
$$t=|r|\sqrt{\frac{n-2}{1-r^2}}t(n-2)$$
(3) Make decisions. According to a given significance level α and degrees of freedom$df=n-2Check$ the t distribution table and find out$t_{a/}(n-2)$ critical value. If$|t|>t_{a/}$, then the null hypothesis H0 is rejected, indicating that there is a significant linear relationship between the two variables in the population.

7. Explain the meaning of regression model, regression equation, and estimated regression equation.

(1) Regression model: For two variables with a linear relationship, a linear equation can be used to express the relationship between them. Describe the dependent variable $How\; y$ depends on the independent variable$x$ and the error term$The\; equation\; for\; \epsilon$ is called a regression model. For a linear regression model involving only one independent variable, it can be expressed as:
$$y=b_0+b_{1}x+e$$

(2) Regression equation: According to the assumptions in the regression model, $The\; expected\; value\; of\; \epsilon$ is equal to 0, so the expected value of y$E\left(and\right)=b_0+b_{1}x$ , that is,$The\; expected\; value\; of\; y$ is$A\; linear\; function\; of\; x$ . Describe the dependent variable$The\; equation\; of\; how\; the\; expected\; value\; of\; y$ depends on the independent variable x is called a regression equation. The unary linear regression equation has the form:
$$E\left(and\right)=b_0+b_{1}x$$

(3) Estimated regression equation: If the parameter in the regression equation $b_0$and $b_1$Known, for a given $The\; value\; of\; x$ , using the formula$E\left(and\right)=b_0+b_{1}x$ can calculate$The\; expected\; value\; of\; y$ . But the overall regression parameter$b_0$and $b_1$are unknown and must be estimated using sample data. Using the sample statistic ${\hat{b}}_0$and ${\hat{b}}_1$Instead of the unknown parameter β 0 {\beta}_0 in the regression equation$b_0$and $b_1$, then the estimated regression equation is obtained. It is an estimate of the regression equation found on the sample data.
For unary linear regression, the estimated regression equation is of the form:
$$\hat{y}={\hat{b}}_0+{\hat{b}}_{1}x$$

where ${\hat{b}}_0$is the intercept of the estimated regression line on the y-axis; ${\hat{b}}_1$is the slope of the straight line, which means $When\; x$ changes by one unit,yyThe mean change in$y\; .$

8. What are the basic assumptions in the linear regression model?

The basic assumptions in the unary linear regression model:

(1) dependent variable $y$ and the independent variable$There\; is\; a\; linear\; relationship\; between\; x$ .
(2) In repeated sampling, the independent variable$The\; value\; of\; x$ is fixed, that is, it is assumed that$x$ is non-random.
(3) Error term$\epsilon$ is a random variable with an expected value of 0, ie$E\left(e\right)=0$ .
(4) For all$x$值，ε \varepsilonThe variance of$\epsilon$$p^2$ are the same.
(5) Error term$\epsilon$ is a random variable that obeys a normal distribution and is independent, that is,$e\sim N(0,p^2)$。

9. Briefly describe the basic principle of parameter least squares estimation.

for $i$ xx_$x$ value, the estimated regression equation can be expressed as:

$$\hat{y}={\hat{b}}_0+{\hat{b}}_{1}x$$

for $x$和yynnof$y$$For\; n$ pairs of observations, there are many straight lines used to describe their relationship. The relationship between the two variables is represented by a straight line closest to each observation point, and it is estimated by minimizing the sum of squared deviations between the observed value and the estimated value. Parameter$b_0$and $b_1$, according to this method to determine the model parameters $b_0$and $b_1$The method is called the least square method, also known as the least square method, it is to make the observed value of the dependent variable $y_i$with estimated value ${\hat{y}}_i$The sum of squared deviations between them reaches the minimum to estimate $b_0$and $b_1$Methods.

10. Explain the meaning of the total sum of squares, regression sum of squares, and residual sum of squares, and explain the relationship between them.

(1) The total sum of squares (SST) is the actual observed value $y_i$and its mean $\bar{y}$For the functional form,
$$SST=\sum (y_i-\bar{y}{)}^2=\sum (y_i-{\hat{y}}_i{)}^2+\sum ({\hat{y}}_i-\bar{y}{)}^2$$

(2) Regression sum of squares (SSR) is each regression value ${\hat{y}}_i$and the mean y of the actual observations $\bar{y}$The sum of squared deviations, that is, $SSR=\sum ({\hat{y}}_i-\bar{y}{)}^2$ , which reflects theyyThe total variation in$y$$x$与yyyycaused by a linear relationship between$y$$The\; variable\; part\; of\; y$ , which can be explained by the regression line$y_i$variation part.

(3) The residual sum of squares (SSE) is the actual observed value $y_i$with regression value yyThe sum of squared deviations of$y$$\sum (y_i-{\hat{y}}_i{)}^2$ , it is except$x$对yyFactors other than the linear influence of$y$$The\; effect\; of\; y$ variation is yi y_ithat cannot be explained by the regression line$y_i$variation part. It is also known as the error sum of squares.

(4) The relationship between the three: total sum of squares (SST) = regression sum of squares (SSR) + residual sum of squares (SSE).

11. Briefly describe the meaning and function of the coefficient of determination.

(1) The meaning of the coefficient of determination
The ratio of the regression sum of squares to the total sum of squares is called the coefficient of determination, denoted as $R_2$，其计算公式为
$$R^2=\frac{SSR}{SST}=\frac{\sum ({\hat{y}}_i-\bar{y}{)}^2}{\sum (y_i-\bar{y}{)}^2}=1-\frac{\sum (y_i-{\hat{y}}_i{)}^2}{\sum (y_i-\bar{y}{)}^2}$$

(2) The role of the coefficient of determination
The coefficient of determination $R_2$It measures how well the regression line fits the observed data. If all the observation points fall on the straight line, the residual sum of squares $SSE＝0,R_2=1$ , the fit is complete; if$y$ changes with$x$ is irrelevant,$x$ doesn't help explain$The\; variation\; of\; y$ , at this time, then$R_2=0$ . Visible$R_2$The value range of is [0,1]. $R_2$The closer it is to 1, the larger the proportion of the regression sum of squares to the total sum of squares is, and the closer the regression line is to each observation point, use xxchange in$x$$The\; more\; the\; part\; of\; the\; y$ value becomes worse, the better the fitting degree of the regression line; otherwise,$R_2$The closer it is to 0, the worse the fit of the regression line is.

12. In regression analysis, what is the role of F test and t test?

The F-test is to test the independent variable $x$ and dependent variable$Is\; the\; linear\; relationship\; between\; y$ significant, or can they use a linear model$y=b_0+b_{1}x+\epsilon$ , which is the test of linear relationship.
The t test is to test whether the independent variable has a significant impact on the dependent variable, that is, the test of the regression coefficient.

13. Briefly explain the role of residual analysis in regression analysis.

The role of residual analysis in regression analysis is as follows:

(1) It is used to judge whether the assumptions about the model are established;
(2) It is used to analyze the outliers in the regression and the observed values ​​that have an impact on the model.