Multiplicity

Definition: Potential inflation of Type I error rates (incorrectly rejecting the null hypothesis) because of multiple statistical testing, for example, multiple outcomes, multiple follow-up time points, or multiple subgroup analyses. To overcome issues with multiplicity, researchers will often apply controlling procedures (e.g., Bonferroni, Holm-Bonferroni; Tukey) that correct the alpha value to control for inflated Type I errors. However, by controlling for Type I errors, one can increase the possibility of Type II errors (i.e., incorrectly accepting the null hypothesis).

Related terms: Alpha, False Discovery Rate, Multiple comparisons problem, Multiple testing, Null Hypothesis Significance Testing (NHST)

References: Sato (1996), & Schultz and Grimes (2005)

Drafted and Reviewed by: Aidan Cashin, Jamie P. Cockcroft, Mahmoud Elsherif, Meng Liu, Charlotte R. Pennington

Note that we are currently working on an automated mechanism to link references cited above with their full-length version that can be found at https://forrt.org/glossary/references with all references used so far.