Researcher degrees of freedom

Last updated on Feb 26, 2026
Also available in: Arabic | German | Turkish
 

Definition: refers to the flexibility often inherent in the scientific process, from hypothesis generation, designing and conducting a research study to processing the data and analyzing as well as interpreting and reporting results. Due to a lack of precisely defined theories and/or empirical evidence, multiple decisions are often equally justifiable. The term is sometimes used to refer to the opportunistic (ab-)use of this flexibility aiming to achieve desired results —e.g., when in- or excluding certain data— albeit the fact that technically the term is not inherently value-laden.

Related terms: Analytic Flexibility, Garden of forking paths, Model uncertainty, Multiverse analysis, *P*\-hacking, Robustness (analyses), Specification curve analysis

References:

  • Gelman, A., & Loken, E. (n.d.). The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time. Retrieved from http://www.stat.columbia.edu/
  • Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632
  • Wicherts, J. M., Veldkamp, C. L., Augusteijn, H. E., Bakker, M., Van Aert, R., & Van Assen, M. A. (2016). Degrees of freedom in planning, running, analyzing, and reporting psychological studies: A checklist to avoid p-hacking. Frontiers in Psychology, 7, 1832. https://doi.org/10.3389/fpsyg.2016.01832

Originally drafted by: Tina Lonsdorf

Reviewed by: Gilad Feldman, Helena Hartmann, Timo Roettger, Robbie C.M. van Aert, FlĂĄvio Azevedo