Optional Stopping

Definition: The practice of (repeatedly) analyzing data during the data collection process and deciding to stop data collection if a statistical criterion (e.g. p-value, or bayes factor) reaches a specified threshold. If appropriate methodological precautions are taken to control the type 1 error rate, this can be an efficient analysis procedure (e.g. Lakens, 2014). However, without transparent reporting or appropriate error control the type 1 error can increase greatly and optional stopping could be considered a Questionable Research Practice (QRP) or a form of p-hacking.

Related terms: P-hacking, Questionable Research Practices or Questionable Reporting Practices (QRPs), Sequential testing

References: Beffara Bret et al. (2021), Lakens (2014), Sagarin et al. (2014), & Schönbrodt et al. (2017)

Drafted and Reviewed by: Brice Beffara Bret, Bettina M. J. Kern, Ali H. Al-Hoorie, Helena Hartmann, Catia M. Oliveira, Sam Parsons

Note that we are currently working on an automated mechanism to link references cited above with their full-length version that can be found at https://forrt.org/glossary/references with all references used so far.