4 Pre-analysis Planning
5 sub-clusters · 42 referencesPre-analysis planning entails laying out a complete methodology and analysis before a study has been undertaken. This facilitates transparency and removes several potential QRPs. When teaching, students should attain knowledge regarding what a pre-registration entails, why it is important to remove potential QRPs and how to address deviations from preregistered plans. There are 5 sub-clusters which aim to further parse the learning and teaching process:
Preregistration
When a researcher preregisters their work, they typically upload a detailed project plan before the start of data collection. This plan includes (but is not limited to) hypotheses, data collection procedures, measures and manipulated variables, and the analysis plan.
- Besançon, L., Nosek, B. A., Haven, T. L., Meyer, M., Dunne, C., & Ghoniem, M. (2024). Merits and Limits of Preregistration for Visualization Research. https://doi.org/10.31219/osf.io/n7ej3
- Brouwers, K., Cooke, A., Chambers, C. D., Henson, R., & Tibon, R. (2020). Evidence for prereg posters as a platform for preregistration. Nature Human Behaviour, 4(9), 884–886. https://doi.org/10.1038/s41562-020-0868-z
- Centre for Open Science. (n.d.). Preregistration. Www.cos.io. https://www.cos.io/initiatives/prereg
- DeHaven, A. (2017). 10 Tips for Making a Great Preregistration. www.cos.io. https://www.cos.io/blog/10-preregistration-tips
- DeHaven, A. (2018). One Preregistration to Rule Them All? Www.cos.io. https://www.cos.io/blog/one-preregistration-rule-them-all
- L. Haven, T., & Van Grootel, Dr. L. (2019). Preregistering qualitative research. Accountability in Research, 26(3), 229–244. https://doi.org/10.1080/08989621.2019.1580147
- Haven, T. L., Errington, T. M., Gleditsch, K. S., van Grootel, L., Jacobs, A. M., Kern, F. G., Piñeiro, R., Rosenblatt, F., & Mokkink, L. B. (2020). Preregistering Qualitative Research: A Delphi Study. International Journal of Qualitative Methods, 19. https://doi.org/10.1177/1609406920976417
- Kirtley, O. J., Lafit, G., Achterhof, R., Hiekkaranta, A. P., & Myin-Germeys, I. (2019). Making the black box transparent: A template and tutorial for (pre-)registration of studies using Experience Sampling Methods (ESM). https://doi.org/10.31234/osf.io/seyq7
- Klonsky, E. D. (2024). Campbell’s Law Explains the Replication Crisis: Pre-Registration Badges Are History Repeating. Assessment, 32(2), 224–234. https://doi.org/10.1177/10731911241253430
- Mertens, G., & Krypotos, A.-M. (2019). Preregistration of analyses of preexisting data. https://doi.org/10.31234/osf.io/ph4q7
- Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600–2606. https://doi.org/10.1073/pnas.1708274114
- Syed, M. (2023). Some Data Indicating that Editors and Reviewers Do Not Check Preregistrations during the Review Process. https://doi.org/10.31234/osf.io/nh7qw
- Van den Akker, O. R., Weston, S., Campbell, L., Chopik, B., Damian, R., Davis-Kean, P., Hall, A., Kosie, J., Kruse, E., Olsen, J., Ritchie, S., Valentine, K., Van ’t Veer, A., & Bakker, M. (2021). Preregistration of secondary data analysis: A template and tutorial. Meta-Psychology, 5. https://doi.org/10.15626/MP.2020.2625
- van ’t Veer, A. E., & Giner-Sorolla, R. (2016). Pre-registration in social psychology—A discussion and suggested template. Journal of Experimental Social Psychology, 67, 2–12. https://doi.org/10.1016/j.jesp.2016.03.004
Purpose of pre-analysis planning
Distinguishing exploratory and confirmatory analyses. One proposed benefit to this is transparency. FORRT does not propose one has more value over the other.
- Dal-Ré, R., Ioannidis, J. P., Bracken, M. B., Buffler, P. A., Chan, A.-W., Franco, E. L., La Vecchia, C., & Weiderpass, E. (2014). Making Prospective Registration of Observational Research a Reality. Science Translational Medicine, 6(224). https://doi.org/10.1126/scitranslmed.3007513
- Davis-Stober, C. P., Sarafoglou, A., Aczel, B., Chandramouli, S. H., Errington, T. M., Field, S. M., Fishbach, A., Freire, J., Ioannidis, J. P. A., Oberauer, K., Pestilli, F., Ressl, S., Schad, D. J., ter Schure, J., Tentori, K., van Ravenzwaaij, D., Vandekerckhove, J., & Gundersen, O. E. (2025). How can we make sound replication decisions? Proceedings of the National Academy of Sciences, 122(5). https://doi.org/10.1073/pnas.2401236121
- Graf, C. (2017). 8 Answers About Registered Reports and Research Preregistration. Wiley Library. www.wiley.com. https://www.wiley.com/network/researchers/being-a-peer-reviewer/8-answers-about-registered-reports-research-preregistration-and-why-both-are-important
- Lakens, D. (2019). The Value of Preregistration for Psychological Science: A Conceptual Analysis. https://doi.org/10.31234/osf.io/jbh4w
- Lin, W., & Green, D. P. (2016). Standard Operating Procedures: A Safety Net for Pre-Analysis Plans. Political Science and Politics, 49(3), 495–500. https://doi.org/10.1017/S1049096516000810
- Nosek, B. A., & Lakens, D. (2014). Registered Reports. Social Psychology, 45(3), 137–141. https://doi.org/10.1027/1864-9335/a000192
- Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600–2606. https://doi.org/10.1073/pnas.1708274114
- Nuzzo, R. (2015). How scientists fool themselves – and how they can stop. Nature, 526(7572), 182–185. https://doi.org/10.1038/526182a
- Wagenmakers, E. J., & Dutilh, G. (2016). Seven selfish reasons for preregistration. APS Observer, 29(9). https://www.psychologicalscience.org/observer/seven-selfish-reasons-for-preregistration
- Wicherts, J. M., Veldkamp, C. L. S., Augusteijn, H. E. M., Bakker, M., van Aert, R. C. M., & van Assen, M. A. L. M. (2016). Degrees of Freedom in Planning, Running, Analyzing, and Reporting Psychological Studies: A Checklist to Avoid p-Hacking. Frontiers in Psychology, 7. https://doi.org/10.3389/fpsyg.2016.01832
Registered reports
A publishing format consisting of a preregistration with a specific journal that undergoes peer review. Specifically, the journal reviews the introduction and methodology and upon in-principle acceptance (IPA, in stage 1), it agrees to publish the study assuming the preregistration is followed and deviations are reported (stage 2 acceptance).
- Alogna, V. K., Attaya, M. K., Aucoin, P., Bahník, Š., Birch, S., Birt, A. R., Bornstein, B. H., Bouwmeester, S., Brandimonte, M. A., Brown, C., Buswell, K., Carlson, C., Carlson, M., Chu, S., Cislak, A., Colarusso, M., Colloff, M. F., Dellapaolera, K. S., Delvenne, J.-F., … Zwaan, R. A. (2014). Registered Replication Report. Perspectives on Psychological Science, 9(5), 556–578. https://doi.org/10.1177/1745691614545653
- Anon. (n.d.). Wiley’s Reviewing Registered Reports. https://authorservices.wiley.com/Reviewers/journal-reviewers/how-to-perform-a-peer-review/reviewing-registered-reports.html
- Chambers, C. D. (2013). Registered Reports: A new publishing initiative at Cortex. Cortex, 49(3), 609–610. https://doi.org/10.1016/j.cortex.2012.12.016
- Chambers, C. D., Dienes, Z., McIntosh, R. D., Rotshtein, P., & Willmes, K. (2015). Registered Reports: Realigning incentives in scientific publishing. Cortex, 66, A1–A2. https://doi.org/10.1016/j.cortex.2015.03.022
- D. Chambers, C., Feredoes, E., D. Muthukumaraswamy, S., & J. Etchells, P. (2014). Instead of “playing the game” it is time to change the rules: Registered Reports at AIMS Neuroscience and beyond. AIMS Neuroscience, 1(1), 4–17. https://doi.org/10.3934/Neuroscience.2014.1.4
- Chambers, C. D., Dienes, Z., McIntosh, R. D., Rotshtein, P., & Willmes, K. (2015). Registered Reports: Realigning incentives in scientific publishing. Cortex, 66, A1–A2. https://doi.org/10.1016/j.cortex.2015.03.022
- Chambers, C. D., & Tzavella, L. (2021). The past, present and future of Registered Reports. Nature Human Behaviour, 6(1), 29–42. https://doi.org/10.1038/s41562-021-01193-7
- Gya, R., Birkeli, K., Dahle, I. J., Foote, C. G., Geange, S. R., Lynn, J. S., Töpper, J. P., Vandvik, V., Zernichow, C., & Jenkins, G. B. (2023). Registered Reports: A new chapter at Ecology & Evolution. Ecology and Evolution, 13(4). Portico. https://doi.org/10.1002/ece3.10023
- Karhulahti, V.-M. (2022). Registered reports for qualitative research. Nature Human Behaviour, 6(1), 4–5. https://doi.org/10.1038/s41562-021-01265-8
- Zotero. (n.d.). OSF Collection of Registered Reports. Zotero. https://www.zotero.org/groups/479248/osf/collections/KEJP68G9/search/right/titleCreatorYear
Study Design
Design elements should be before data collection so that confirmatory claims are credible and deviations are interpretable. This includes choosing appropriate sample sizes, conditions, and measures prior to registration, and planning how to handle deviations
- Lakens, D. (2023). When and How to Deviate from a Preregistration. https://doi.org/10.31234/osf.io/ha29k
Understanding the types of preregistration and writing one.
Practical materials and resources to move beyond the theory of preregistration, into conducting a preregistered study.
- Call, M., DeHaven, A. C., Pfeiffer, N., Mellor, D. T., & Lowrey, O. (2023, October 19). Templates of OSF registration forms. OSF. https://osf.io/zab38/
- Centre for Open Science. (n.d.). Registered Reports: Peer review before results are known to align scientific values and practices. Www.cos.io. https://www.cos.io/initiatives/registered-reports
- Lakens, D. (2023). When and How to Deviate from a Preregistration. https://doi.org/10.31234/osf.io/ha29k
- Mertens, G., & Krypotos, A.-M. (2019). Preregistration of analyses of preexisting data. https://doi.org/10.31234/osf.io/ph4q7
- Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018). The preregistration revolution. Proceedings of the National Academy of Sciences, 115(11), 2600–2606. https://doi.org/10.1073/pnas.1708274114
- Stewart, S. L. K., Rinke, E. M., McGarrigle, R., Lynott, D., Lunny, C., Lautarescu, A., Galizzi, M. M., Farran, E. K., & Crook, Z. (2020). Pre-registration and Registered Reports: a Primer from UKRN. https://doi.org/10.31219/osf.io/8v2n7
- van den Akker, O. R., Peters, G.-J. Y., Bakker, C. J., Carlsson, R., Coles, N. A., Corker, K. S., Feldman, G., Moreau, D., Nordström, T., Pickering, J. S., Riegelman, A., Topor, M. K., van Veggel, N., Yeung, S. K., Call, M., Mellor, D. T., & Pfeiffer, N. (2023). Increasing the transparency of systematic reviews: presenting a generalized registration form. Systematic Reviews, 12(1). https://doi.org/10.1186/s13643-023-02281-7