Topics

• Next steps after ANOVA

• Individual vs. family-wise Type I error

• Multiple comparisons using Bonferroni correction

Aldrin in the Wolf River

• The Wolf River in Tennessee flows past an abandoned site once used by the pesticide industry for dumping wastes, including chlordane (pesticide), aldrin, and dieldrin (both insecticides).

• These highly toxic organic compounds can cause various cancers and birth defects.

Aldrin in the Wolf River

• The standard methods to test whether these substances are present in a river is to take samples at six-tenths depth.

• These compounds are denser than water and their molecules tend to stick to particles of sediment, they are more likely to be found in higher concentrations near the bottom than near mid-depth.

• We will compare mean concentration levels (in nanograms per liter) for three depths.

ANOVA

ANOVA

The p-value is very small (≈0), so we reject H0. The data provide sufficient evidence that at least one depth level has a mean aldrin concentration that differs from the others.

Difference in means

We can use confident intervals to estimate the difference between the means, μi−μj for each pair of groups

If we have K groups, we will make (K2)=K(K−1)/2 such comparisons

Comparisions for Aldrin data set

There are 3 depth levels in our data, so we can make (32)=3(3−1)/2=3 comparisons

(ˉymiddepth−ˉybottom)±t∗×√MSWithin(1nmiddepth+1nbottom)

(ˉysurface−ˉybottom)±t∗×√MSWithin(1nsurface+1nbottom)

Individual vs. Family-wise Type I Error

Type I error: Incorrectly reject H0.

• In our example, incorrectly reject the null hypothesis that mean aldrin concentration levels are equal
• Based on our confidence interval, we incorrectly conclude there is a difference in the mean aldrin concentration for the two groups

Individual Type I error: incorrectly reject H0 for one specific comparison of group means

Family-wise Type I error: Incorrectly reject H0 for at least one comparison of group means

Multiple Comparisons

• The probability of making an individual Type I error is α=1−C, where C is the confidence level

• Even if the probability of making an individual Type I error is low, the probability of making a family-wise Type I error becomes much larger when we make multiple comparisons

xkcd "Significant"

Correcting for multiple comparisons

When we make multiple comparisons, we will select the critical value t∗ to control for the probability of making a family-wise Type I error

Bonferroni correction

Goal: Choose the critical value t∗ such that the probability of making a family-wise Type I error is α.

To do so, we will choose t∗ such that the probability of making an individual Type I error is αm, where m is the number of comparisons

In other words, we will find t∗ that corresponds to a confidence level of 1−α/m.

Comparisons for the Aldrin data set

We want the probability of making a family-wise Type I error to be α=0.05.

We are making 3 comparisons. Therefore, we want probability of making an individual Type I error to be α/m=0.05/3.

Pairwise comparisions in R

library(pairwiseCI)
pairwiseCI(aldrin ~ depth, data = aldrin, 
           conf.level = 1- 0.05/3, var.equal = TRUE) %>%
  kable(digits = 3)

Comparing Aldrin concentrations

Based on this, we see there is a statistically significant difference between the mean aldrin concentration at the surface and at the bottom. More specifically, we are 98.3% confident that the mean aldrin level is about 0.412 to 3.268 nanograms per liter lower at the surface than at the bottom.

Recap

• Next steps after ANOVA

• Individual vs. family-wise Type I error

• Multiple comparisons using Bonferroni correction