Analysis of variance (1)
1. ANOVA model fitting
1.1 aov() function
Syntax: aov(formula, data = dataframe)
Special symbols in R expressions:
Expressions for common research designs in R language:
1.2 The order of terms in an expression
The order of terms in an expression matters in two cases, for example, y ~ A B does not give the same result as y ~ B A:
- There are multiple factors and an unbalanced design (that is, the number of samples in each group is different)
- There are covariates(There will be differences between each sample, and the variables that cause these differences are called covariates)
R language defaults to type I (sequential type)
. Another example: y~A+B+A:B:
- The effect of A on y
- The effect of B on y while controlling A
- When controlling the main effects of A and B, the interaction effect of A and B
Effect of the order of the factors on the result:
The more uneven the sample size, the more the order of the effect terms affects the results. Therefore, the more basic factors should be placed in front of the expression, that is, the covariates should be placed before the main variables, followed by the two-factor interaction term, and then the three-factor interaction term.
For the main effect, the more basic variables should be placed in front of the expression, for example, gender should be placed in front of the treatment.
The Anova() function in the car package provides methods using Type 2 and Type 3, and the aov() function uses Type 1 methods.
2. One-Way ANOVA
One-way ANOVA compares the means of the dependent variable in two or more groups defined by categorical factors. Taking the cholesterol dataset in the multcomp package as an example (taken from Westfall, Tobia, Rom, Hochberg, 1999), 50 patients were treated with one of the five treatments of cholesterol-lowering drug therapy (trt). Three of the treatment conditions use the same drug, which are 20mg once a day (1time), 10mg twice a day (2times) and 5mg four days a day (4times). The remaining two modalities (drugD and drugE) represent drug candidates. Which drug therapy lowered cholesterol (response variable) the most?
install.packages('multcomp')
library(multcomp)
attach(cholesterol) #打开连接：直接使用其中的变量名而不需要使用符号
colnames(cholesterol)
table(trt)
## 计算各组平均值mean
## aggregate()函数是用于对数据进行分组统计的函数:
# aggregate()函数是用于对数据进行分组统计的函数。
# response是一个向量，trt是一个因子变量，表示不同的处理组。
# aggregate(response, by=list(trt), FUN=mean)的作用是计算response向量中每个处理组的平均值
aggregate(response, by=list(trt), FUN=mean)
response
## 计算各组标准差sd
aggregate(response, by=list(trt), FUN=sd)
## 检验组间差异
fit <- aov(response ~ trt)
summary(fit)
#自由度 平方和 均方和 F值 P值
result:
> aggregate(response, by=list(trt), FUN=mean)
Group.1 x
1 1time 5.78197
2 2times 9.22497
3 4times 12.37478
4 drugD 15.36117
5 drugE 20.94752
> aggregate(response, by=list(trt), FUN=sd)
Group.1 x
1 1time 2.878113
2 2times 3.483054
3 4times 2.923119
4 drugD 3.454636
5 drugE 3.345003
> summary(fit)
Df Sum Sq Mean Sq F value Pr(>F)
trt 4 1351.4 337.8 32.43 9.82e-13 ***
Residuals 45 468.8 10.4
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
The F-test of ANOVA for treatment (trt) was very significant (p<0.0001), indicating that the five treatments had different effects.
2.1 Draw the graph of each group mean and its confidence interval
## 绘制各组均值及其置信区间的图形
library(gplots)
plotmeans(response ~ trt, xlab="Treatment", ylab="Response",
main="Mean Plot\nwith 95% CI")
detach(cholesterol) # 关闭连接
Degree of freedom: Considering a sample with a sample size of n (n numbers), if we know the sample mean, then we only need to know the first n-1 numbers to get all the information, because the last number can be calculated with the help of the mean out. so,Given the sample mean (constraints), we have reduced one degree of freedom based on the original n numbers, and the information provided by the sample is only n-1 independent parts.
2.2 Multiple comparisons - TukeyHSD
While the ANOVA's F-test for each treatment shows that the five drug treatments work differently, it doesn't tell you which treatment is different from the others. Multiple comparisons can solve this problem, where the TukeyHSD() function provides a pairwise test for the difference in the means of the groups.
TukeyHSD(fit)
result:
Tukey multiple comparisons of means
95% family-wise confidence level
Fit: aov(formula = response ~ trt)
$trt
diff lwr upr p adj
2times-1time 3.44300 -0.6582817 7.544282 0.1380949
4times-1time 6.59281 2.4915283 10.694092 0.0003542
drugD-1time 9.57920 5.4779183 13.680482 0.0000003
drugE-1time 15.16555 11.0642683 19.266832 0.0000000
4times-2times 3.14981 -0.9514717 7.251092 0.2050382
drugD-2times 6.13620 2.0349183 10.237482 0.0009611
drugE-2times 11.72255 7.6212683 15.823832 0.0000000
drugD-4times 2.98639 -1.1148917 7.087672 0.2512446
drugE-4times 8.57274 4.4714583 12.674022 0.0000037
drugE-drugD 5.58635 1.4850683 9.687632 0.0030633
It can be seen that the mean difference between 1time and 2times is not significant (p=0.138), while the difference between 1time and 4times is very significant (p<0.001).
2.2.1 Pairwise comparison plot
The first par statement is used to rotate the axis label, and the second is used to increase the area of the left border, which can make the label placement more beautiful.The treatments whose confidence interval contains 0 in the graph indicate that the difference is not significant (p>0.5)
In the R language, par(las=2) is used to set the label direction of the drawing area. where las is an integer specifying the orientation of the label.las=0 indicates that the label is parallel to the axis，las=1 indicates that the label is perpendicular to the axis，las=2 indicates that the label is perpendicular to the axis and the text is written from right to left。
par(las=2)
par(mar=c(5,8,4,2))
plot(TukeyHSD(fit))
The result display:
2.2.2 Multiple mean comparison - TukeyHSD (easier to understand)
The glht() function in the multcomp package provides a more comprehensive method for multiple mean comparisons, both for linear models) and for generalized linear models. The following code reproduces the Tukey HSD test and displays the results with a different graph.
library(multcomp)
par(las=0)
par(mar=c(5,4,6,2))
tuk <- glht(fit, linfct=mcp(trt="Tukey"))
plot(cld(tuk, level=.05),col="lightgrey") #cld()函数中的level选项设置了使用的显著水平0.05
Result presentation:
Image interpretation:
Groups with the same letter (represented by boxplots) indicate that the difference in means is not significant. It can be seen that the difference between 1time and 2times is not significant (they have the same letter a), the difference between 2times and 4times is not significant (they have the same letter b), and the difference between 1time and 4times is significant (they have no common letter). Personally, I think this graph is easier to understand than the pairwise comparison graph in 2.2.1.
2.3 Assumptions for the evaluation test
2.3.1 Normality test
Our confidence in the results depends on doing statistical testsThe extent to which the data satisfy the assumptions. In one-way ANOVA, we assume that the dependent variableobbey normal distribution，All groups have equal variance. A QQ plot can be used to test the normality assumption:
library(car)
# qqPlot()要求用lm()拟合
qqPlot(lm(response ~ trt, data=cholesterol),
simulate=TRUE, main="Q-Q Plot", labels=FALSE)
QQ plot:
The data falls within the 95% confidence interval, indicating that the normality assumption is met.
2.3.2 Homogeneity of variance test
library(car)
qqPlot(lm(response ~ trt, data=cholesterol),
simulate=TRUE, main="Q-Q Plot", labels=FALSE)
The result display:
Bartlett test of homogeneity of variances
data: response by trt
Bartlett's K-squared = 0.57975, df = 4, p-value = 0.9653
There are other tests such as Fligner-Killeen test (fligner.test() function) and Brown-Forsythe test (hov() function in HH package), which obtain the same results as Bartlett's test.
2.3.3 Outlier test
Outliers: Points that predict poorly, with enlarged residuals.
The presence or absence of outliers is judged based on the significance of the largest residual value:
- If it is not significant Bonferonni P>0.05, it indicates that there is no outlier;
- If it is significant Bonferonni P<0.05, it indicates that the point with the largest residual value is an outlier and needs to be deleted, and then the outlier test is performed again on the fitting model after deleting this point.
ANOVA is very sensitive to outliers. The outlierTest() function in the car package can be used to detect outliers:
library(car)
outlierTest(fit)
The result display:
No Studentized residuals with Bonferroni p < 0.05
Largest |rstudent|:
rstudent unadjusted p-value Bonferroni p
19 2.251149 0.029422 NA
Analysis of Results:
This result is the output of a statistical test on a particular regression analysis. For the 19th data point, the test computed a studentized residual of 2.251, corresponding to an uncorrected p-value of 0.029422. However, no studentized residual values with a Bonferroni-corrected p-value less than 0.05 were found, so the output reads "No studentized residual with Bonferroni p < 0.05". From this, it can be concluded that the studentized residual value for this data point is not significant, that is, not statistically significant(Not statistically significant means that the result is not strong enough to support the hypothesis. In this case, we cannot reject the null hypothesis. In the example, the Bonferroni p-value for no studentized residual is less than 0.05, which means that there is no outliers).
From the output results, there is no evidence that there are outliers in the cholesterol data (NA will be generated when p>1). So based on the QQ plot, the Bartlett test and the outlier test, the data seem to fit well with the ANOVA model. These methods in turn enhance our confidence in the results obtained.
3. One-way covariance analysis
One-way analysis of covariance (ANCOVA) extends one-way analysis of variance (ANOVA) to include one or more quantitative covariates . The examples below are from the litter dataset in the multcomp package. Pregnant mice were divided into four groups, and each group received different doses (0, 5, 50 or 500) of the drug treatment. The mean weight of pups born was the dependent variable and gestation time was the covariate .
#单因素协方差分析
data(litter, package="multcomp")
attach(litter)
table(dose)
aggregate(weight, by=list(dose), FUN=mean)
fit <- aov(weight ~ gesttime + dose) # 注意aov()使用的是类型1
summary(fit)
The result display:
> table(dose)
dose
0 5 50 500
20 19 18 17
> aggregate(weight, by=list(dose), FUN=mean)
Group.1 x
1 0 32.30850
2 5 29.30842
3 50 29.86611
4 500 29.64647
> summary(fit)
Df Sum Sq Mean Sq F value Pr(>F)
gesttime 1 134.3 134.30 8.049 0.00597 **
dose 3 137.1 45.71 2.739 0.04988 *
Residuals 69 1151.3 16.69
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Result analysis:
Using the table() function, it can be seen that the number of pups born under each dose is different: 20 pups were born when the dose was 0 (no drug), and 17 pups were born when the dose was 500. Then use the aggregate() function to obtain the average value of each group, and it can be found that the average weight of pups in the untreated group is the highest (32.3). The F-test of ANCOVA showed that:
(a) the pregnancy time was related to the birth weight of the pups;
(b) the drug dose was related to the birth weight when the pregnancy time was controlled.
Controlling for pregnancy time, it was indeed found that the mean birth weight of pups was different for each drug dose.
3.1 Remove the influence of covariates
Because of the use of covariates, you may want to obtain adjusted group means, that is, group means after removing the effects of covariates. Adjusted means can be calculated using the effects() function from the effects package :
library(effects)
effect("dose", fit)
result:
dose effect
dose
0 5 50 500
32.35367 28.87672 30.56614 29.33460
The adjusted mean is similar to the unadjusted mean obtained by the aggregate() function, but not in all cases. In summary, the effects package provides a powerful way to compute adjusted means for complex research designs and to visualize the results.
3.2 Multiple comparisons of user-defined controls (User-defined Contrasts)
The dose F-test, while showing different mean pup weights for the different treatments, did not tell us which treatment was different from the others. Again, we use the multcomp package to perform pairwise comparisons of all means. In addition, the multcomp package can also be used to test user-defined mean assumptions. Assuming that you are interested in whether the effect of the non-medicated condition is different from that of the other three drugged conditions, you can use the following code to achieve it:
library(multcomp)
contrast <- rbind("no drug vs. drug" = c(3, -1, -1, -1)) #rbind表示“row bind”，即“按行绑定”，其中每个对象应该具有相同的列数和相同的类型
summary(glht(fit, linfct=mcp(dose=contrast)))
c(3, -1, -1, -1) in rbind("no drug vs. drug" = c(3, -1, -1, -1)) is a numeric vector of length 4, where the One value (ie 3) means "specific gravity of uninjected drug", and the following three values (ie -1, -1, -1) indicate "specific gravity of three doses". This vector is used to define a comparison vector that combines the three doses into one group and compares it to the no-drug injection group.
result:
Simultaneous Tests for General Linear Hypotheses
Multiple Comparisons of Means: User-defined Contrasts
Fit: aov(formula = weight ~ gesttime + dose)
Linear Hypotheses:
Estimate Std. Error t value Pr(>|t|)
no drug vs. drug == 0 8.284 3.209 2.581 0.012
no drug vs. drug == 0 *
---
Signif. codes:
0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
(Adjusted p values reported -- single-step method)
Control c(3, -1, -1, -1) sets the means of the first group and the other three groups for comparison. The t statistic (2.581) of the hypothesis test is significant at the p<0.05 level, therefore, it can be concluded that the birth weight of the non-drug group is higher than that of other drug-treated conditions.
3.3 Assumptions for the evaluation test
ANCOVA is the same as ANOVA, both require normality and homoscedasticity assumptions, and ANCOVA also assumes the same regression slope . In this example, it is assumed that the regression slopes for birthweight by gestational time are the same for all four treatment groups. The homogeneity of the regression slopes can be tested when the ANCOVA model includes the pregnancy time × dose interaction term. A significant interaction effect implies that the relationship between time and pup birth weight depends on the level of drug dose.
library(multcomp)
fit2 <- aov(weight ~ gesttime*dose, data=litter)
summary(fit2)
result:
Df Sum Sq Mean Sq F value Pr(>F)
gesttime 1 134.3 134.30 8.289 0.00537 **
dose 3 137.1 45.71 2.821 0.04556 *
gesttime:dose 3 81.9 27.29 1.684 0.17889
Residuals 66 1069.4 16.20
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
It can be seen that the interaction effect is not significant, supporting the assumption of equal slopes. If the assumption is not true, one can try to transform the covariate or dependent variable, or use a model that can explain each slope independently, or use a nonparametric ANCOVA method that does not require the assumption of homogeneity of regression slopes. The sm.ancova() function in the sm package provides an example for the latter.
In short, if the interaction effect is not significant , it means that the influence of the independent variable on the dependent variable is linear, and a unified regression equation can be used to describe the relationship between them.
3.4 Results Visualization
The ancova() function in the HH package can plot the relationship between dependent variables, covariates and factors:
library(HH)
ancova(weight ~ gesttime + dose, data=litter)
result:
Analysis of Variance Table
Response: weight
Df Sum Sq Mean Sq F value Pr(>F)
gesttime 1 134.30 134.304 8.0493 0.005971 **
dose 3 137.12 45.708 2.7394 0.049883 *
Residuals 69 1151.27 16.685
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
It can be seen from the figure that the regression lines using pregnancy time to predict birth weight are parallel to each other, but the intercept term is different. As gestation time increases, pup birth weights also increase. In addition, it can also be seen that the 0-dose group has the largest intercept item, and the 5-dose group has the smallest intercept item. Because of your settings above, the lines will remain parallel,With ancova(weight ~ gesttime*dose), the resulting graph will allow the slope and intercept terms to vary by group, which is useful for visualizing instances where regression slope homogeneity is violated.
4. Two-way analysis of variance (ANOVA)
In a two-way ANOVA, subjects were assigned to two-factor cross-category groups. Taking the ToothGrowth data set in the basic installation as an example, 60 guinea pigs were randomly assigned to two feeding methods (orange juice or vitamin C), and there were three levels of ascorbic acid in each feeding method (0.5mg, 1mg or 2mg). Each treatment combination was assigned 10 guinea pigs. Tooth length was the dependent variable.
#双因素方差分析
attach(ToothGrowth)
table(supp, dose)
head(ToothGrowth)
aggregate(len, by=list(supp, dose), FUN=mean)
aggregate(len, by=list(supp, dose), FUN=sd)
dose <- factor(dose)
fit <- aov(len ~ supp*dose) # 双因素影响
summary(fit)
detach(ToothGrowth)
result:
> head(ToothGrowth)
len supp dose
1 4.2 VC 0.5
2 11.5 VC 0.5
3 7.3 VC 0.5
4 5.8 VC 0.5
5 6.4 VC 0.5
6 10.0 VC 0.5
> aggregate(len, by=list(supp, dose), FUN=mean)
Group.1 Group.2 x
1 OJ 0.5 13.23
2 VC 0.5 7.98
3 OJ 1.0 22.70
4 VC 1.0 16.77
5 OJ 2.0 26.06
6 VC 2.0 26.14
> aggregate(len, by=list(supp, dose), FUN=sd)
Group.1 Group.2 x
1 OJ 0.5 4.459709
2 VC 0.5 2.746634
3 OJ 1.0 3.910953
4 VC 1.0 2.515309
5 OJ 2.0 2.655058
6 VC 2.0 4.797731
> summary(fit)
Df Sum Sq Mean Sq F value Pr(>F)
supp 1 205.4 205.4 15.572 0.000231 ***
dose 2 2426.4 1213.2 92.000 < 2e-16 ***
supp:dose 2 108.3 54.2 4.107 0.021860 *
Residuals 54 712.1 13.2
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Result analysis:
The preprocessing of the table statement indicates that the design is a balanced design (the sample size in each design unit is the same), and the mean value and standard deviation of each unit can be obtained by the aggregate statement processing. The dose variable is transformed into a factor variable so that the aov() function treats it as a grouping variable rather than a numerical covariate. Using the summary() function to get the variance analysis table, we can see that the main effects (supp and dose) and the interaction effects are very significant.
4.1 Results Visualization
available hereinteraction.plot() functionTo demonstrate the interaction effect of two-way ANOVA.
interaction.plot(dose, supp, len, type="b",
col=c("red","blue"), pch=c(16, 18),
main = "Interaction between Dose and Supplement Type")
The graph shows the mean tooth length of guinea pigs fed various doses:
Use the plotmeans() function in the gplots package to visualize interaction effects:
library(gplots)
plotmeans(len ~ interaction(supp, dose, sep=" "),
connect=list(c(1,3,5),c(2,4,6)),
col=c("red", "darkgreen"),
main = "Interaction Plot with 95% CIs",
xlab="Treatment and Dose Combination")
The graph shows the mean, error bars (95% confidence interval) and sample size:
Use the interaction2wt() function in the HH package to visualize the results. The graph shows both the main effect and the interaction effect of the factorial design in any order:
library(HH)
interaction2wt(len~supp*dose)
Main and interaction effects for the ToothGrowth dataset:
All three graphs above show that tooth length increases with increasing doses of ascorbic acid in orange juice (QJ) and vitamin C (VC). For the 0.5 mg and 1 mg doses, orange juice promoted tooth growth more than vitamin C; for the 2 mg dose of ascorbic acid, tooth length growth was the same for both feeding methods. Among the three drawing methods, I recommend the interaction2wt() function in the HH package , because it can display the main effect (box plot) and interaction effect of any complexity design (two-factor ANOVA, three-factor ANOVA, etc.) .
Model hypothesis testing and comparison of means are not covered here as they are just a natural extension of the previous approach. Also, the design is balanced, so there is no need to worry about effect order.