References

Adler, S. J., L. Röseler, and M. K. Schöniger. 2023. “A Toolbox to Evaluate the Trustworthiness of Published Findings.” Journal of Business Research 167: 114189. https://doi.org/10.1016/j.jbusres.2023.114189.

Ankel-Peters, J., A. Brodeur, A. Dreber, M. Johannesson, F. Neubauer, and J. Rose. 2025. “A Protocol for Structured Robustness Reproductions and Replicability Assessments.” Q Open, qoaf004. https://doi.org/10.1093/qopen/qoaf004.

Ankel-Peters, J., N. Fiala, and F. Neubauer. 2023. “Do Economists Replicate?” Journal of Economic Behavior & Organization 212: 219–32. https://doi.org/10.1016/j.jebo.2023.05.009.

Bartoš, F., and U. Schimmack. 2022. “Z-Curve 2.0: Estimating Replication Rates and Discovery Rates.” Meta-Psychology 6. https://doi.org/10.15626/MP.2021.2720.

Baumeister, R. F., D. M. Tice, and B. J. Bushman. 2022. “A Review of Multisite Replication Projects in Social Psychology: Is It Viable to Sustain Any Confidence in Social Psychology’s Knowledge Base?” Perspectives on Psychological Science 18 (4): 912–35. https://doi.org/10.1177/17456916221121815.

Baumeister, R. F., and K. D. Vohs. 2016. “Misguided Effort with Elusive Implications.” Perspectives on Psychological Science 11 (4): 574–75. https://doi.org/10.1177/1745691616652878.

Bekkers, R. 2024. “Replication Value: A Comment and Alternative.” https://doi.org/10.31234/osf.io/uj5g7.

Bennett, E. A. 2021. “Open Science from a Qualitative, Feminist Perspective: Epistemological Dogmas and a Call for Critical Examination.” Psychology of Women Quarterly 45 (4): 448–56. https://doi.org/10.1177/03616843211036460.

Berinsky, A. J., J. N. Druckman, and T. Yamamoto. 2021. “Publication Biases in Replication Studies.” Political Analysis 29 (3): 370–84. https://doi.org/10.1017/pan.2020.34.

Berkeley Initiative for Transparency in the Social Sciences. 2020. “Guide for Advancing Computational Reproducibility in the Social Sciences.” https://bitss.github.io/ACRE/.

Block, J., and A. Kuckertz. 2018. “Seven Principles of Effective Replication Studies: Strengthening the Evidence Base of Management Research.” Management Review Quarterly 68 (4): 355–59. https://doi.org/10.1007/s11301-018-0149-3.

Borgstede, M., and M. Scholz. 2021. “Quantitative and Qualitative Approaches to Generalization and Replication–a Representationalist View.” Frontiers in Psychology 12: 605191. https://doi.org/10.3389/fpsyg.2021.605191.

Boyce, V., B. Prystawski, A. B. Abutto, E. M. Chen, Z. Chen, H. Chiu, and M. C. and Frank. 2024. “Estimating the Replicability of Psychology Experiments After an Initial Failure to Replicate,” May. https://doi.org/10.31234/osf.io/an3yb.

Brandt, M. J., H. IJzerman, A. Dijksterhuis, F. J. Farach, J. Geller, R. Giner-Sorolla, and A. and Van’t Veer. 2014. “The Replication Recipe: What Makes for a Convincing Replication?” Journal of Experimental Social Psychology 50: 217–24. https://doi.org/10.1016/j.jesp.2013.10.005.

Brodeur, A., A. Dreber, F. Hoces de la Guardia, and E. Miguel. 2024. “Reproduction and Replication at Scale.” Nature Human Behaviour 8 (1): 2–3. https://doi.org/10.1038/s41562-023-01807-2.

Bryan, C. J., D. S. Yeager, and J. M. O’Brien. 2019. “Replicator Degrees of Freedom Allow Publication of Misleading Failures to Replicate.” Proceedings of the National Academy of Sciences 116 (51): 25535–45. https://doi.org/10.1073/pnas.1910951116.

Calder, B. J., L. W. Phillips, and A. M. Tybout. 1981. “Designing Research for Application.” Journal of Consumer Research 8 (2): 197–207. https://doi.org/10.1086/208856.

Carter, E. C., F. D. Schönbrodt, W. M. Gervais, and J. Hilgard. 2019. “Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods.” Advances in Methods and Practices in Psychological Science 2 (2): 115–44. https://doi.org/10.1177/2515245919847196.

Chartrand, T. L., and J. A. Bargh. 1999. “The Chameleon Effect: The Perception–Behavior Link and Social Interaction.” Journal of Personality and Social Psychology 76 (6): 893. https://doi.org/10.1037/0022-3514.76.6.893.

Clarke, B., P. Y. (K.) Lee, S. R. Schiavone, M. Rhemtulla, and S. Vazire. 2024. “The Prevalence of Direct Replication Articles in Top-Ranking Psychology Journals.” American Psychologist. https://doi.org/10.1037/amp0001385.

Cole, N. L., S. Ulpts, A. Bochynska, E. Kormann, M. Good, B. Leitner, and T. Ross-Hellauer. 2024. “Reproducibility and Replicability of Qualitative Research: An Integrative Review of Concepts, Barriers and Enablers.” https://doi.org/10.31222/osf.io/n5zkw_v1.

Cortina, J. M., T. Köhler, and L. C. Aulisi. 2023. “Current Reproducibility Practices in Management: What They Are Versus What They Could Be.” Journal of Management Scientific Reports 1 (3-4): 171–205. https://doi.org/10.1177/27550311231202696.

Cowan, N., C. Belletier, J. M. Doherty, A. J. Jaroslawska, S. Rhodes, A. Forsberg, M. Naveh-Benjamin, P. Barrouillet, V. Camos, and R. H. Logie. 2020. “How Do Scientific Views Change? Notes from an Extended Adversarial Collaboration.” Perspectives on Psychological Science 15 (4): 1011–25. https://doi.org/10.1177/1745691620906415.

Dreber, A., and M. Johannesson. 2024. “A Framework for Evaluating Reproducibility and Replicability in Economics.” Economic Inquiry. https://doi.org/10.1111/ecin.13244.

Dunlap, K. 1926. “The Experimental Methods of Psychology.” In Psychologies of 1925, edited by C. Murchison, 331–51. Clark University Press. https://doi.org/10.1037/11020-022.

Errington, T. M., M. Mathur, C. K. Soderberg, A. Denis, N. Perfito, E. Iorns, and B. A. Nosek. 2021. “Investigating the Replicability of Preclinical Cancer Biology.” eLife 10: e71601. https://doi.org/10.7554/eLife.71601.

Feldman, G. 2025. “The Value of Replications Goes Beyond Replicability and Is Associated with the Value of the Research It Replicates: Commentary on Isager Et Al., 2021.” Meta Psychology 9. https://doi.org/10.15626/MP.2024.4326.

Field, S. M., R. Hoekstra, L. Bringmann, and D. van Ravenzwaaij. 2019. “When and Why to Replicate: As Easy as 1, 2, 3?” Collabra: Psychology 5 (1): 46. https://doi.org/10.1525/collabra.218.

Francis, G. 2012. “Too Good to Be True: Publication Bias in Two Prominent Studies from Experimental Psychology.” Psychonomic Bulletin & Review 19: 151–56. https://doi.org/10.3758/s13423-012-0227-9.

Friese, M., D. D. Loschelder, K. Gieseler, J. Frankenbach, and M. Inzlicht. 2019. “Is Ego Depletion Real? An Analysis of Arguments.” Personality and Social Psychology Review 23 (2): 107–31. https://doi.org/10.1177/1088868318762183.

Hardwicke, T. E., and E. J. Wagenmakers. 2023. “Reducing Bias, Increasing Transparency and Calibrating Confidence with Preregistration.” Nature Human Behaviour 7 (1): 15–26. https://doi.org/10.1038/s41562-022-01497-2.

Hawkins, R. X., E. N. Smith, C. Au, J. M. Arias, R. Catapano, E. Hermann, and M. C. and Frank. 2018. “Improving the Replicability of Psychological Science Through Pedagogy.” Advances in Methods and Practices in Psychological Science 1 (1): 7–18. https://doi.org/10.1177/2515245917740427.

Heathers, J. 2025. “An Introduction to Forensic Metascience.” https://doi.org/10.5281/zenodo.14871843.

Heirene, R., D. LaPlante, E. Louderback, B. Keen, M. Bakker, A. Serafimovska, and S. Gainsbury. 2024. “Preregistration Specificity and Adherence: A Review of Preregistered Gambling Studies and Cross-Disciplinary Comparison.” Meta-Psychology 8. https://doi.org/10.15626/MP.2021.2909.

Held, L., S. Pawel, and C. Micheloud. 2024. “The Assessment of Replicability Using the Sum of p-Values.” Royal Society Open Science 11 (8): 240149. https://doi.org/10.1098/rsos.240149.

Henriques, S. O., N. Rzayeva, S. Pinfield, and L. Waltman. 2023. “Preprint Review Services: Disrupting the Scholarly Communication Landscape?” https://doi.org/10.31235/osf.io/8c6xm.

Heroux, Michael A., Lorena A. Barba, Manish Parashar, Victoria Stodden, and Michela Taufer. 2018. “Toward a Compatible Reproducibility Taxonomy for Computational and Computing Sciences.” https://doi.org/10.2172/1481626.

Höffler, J. H. 2017. “ReplicationWiki: Improving Transparency in Social Sciences Research.” D-Lib Magazine 23 (3): 1. https://doi.org/10.1045/march2017-hoeffler.

Huang, F. L., and A. B. Huang. 2024. “Replication Studies Using Secondary or Nonexperimental Datasets.” School Psychology Review, 1–15. https://doi.org/10.1080/2372966X.2024.2346781.

Hüffmeier, J., J. Mazei, and T. Schultze. 2016. “Reconceptualizing Replication as a Sequence of Different Studies: A Replication Typology.” Journal of Experimental Social Psychology 66: 81–92. https://doi.org/10.1016/j.jesp.2015.09.009.

Hummel, T., and J. Manner. 2024. “A Literature Review on Reproducibility Studies in Computer Science.” In Proceedings of the 16th ZEUS Workshop on Services and Their Composition (ZEUS 2024)(CEUR). Vol. 3673.

Isager, P. M., R. C. M. van Aert, Š. Bahník, M. J. Brandt, K. A. DeSoto, R. Giner-Sorolla, J. I. Krueger, et al. 2023. “Deciding What to Replicate: A Decision Model for Replication Study Selection Under Resource and Knowledge Constraints.” Psychological Methods 28 (2): 438–51. https://doi.org/10.1037/met0000438.

Jacowitz, K. E., and D. Kahneman. 1995. “Measures of Anchoring in Estimation Tasks.” Personality and Social Psychology Bulletin 21 (11): 1161–66. https://doi.org/10.1177/01461672952111004.

Jekel, M., S. Fiedler, R. Allstadt Torras, D. Mischkowski, A. R. Dorrough, and A. Glöckner. 2020. “How to Teach Open Science Principles in the Undergraduate Curriculum—the Hagen Cumulative Science Project.” Psychology Learning & Teaching 19 (1): 91–106. https://doi.org/10.1177/1475725719868149.

Kamermans, K. L., L. Dudda, T. Daikoku, and S. Verheyen. 2025. “The Is-Ought Problem in Deciding What to Replicate: Which Motives Guide Current Replication Practices?” https://doi.org/10.31234/osf.io/6xdy2_v2.

Karhulahti, V., M. Martončik, and M. Adamkovic. 2024. “Pre-Replication in Meaningful Science.” https://doi.org/10.31234/osf.io/5gn7m.

King, G. 1995. “Replication, Replication.” PS: Political Science & Politics 28 (3): 444–52. https://doi.org/10.2307/420301.

Klein, R. A., K. A. Ratliff, M. Vianello, R. B. Adams Jr, Š. Bahník, M. J. Bernstein, and B. A. and Nosek. 2014. “Investigating Variation in Replicability.” Social Psychology. https://doi.org/10.1027/1864-9335/a000178.

Köhler, T., and J. M. Cortina. 2021. “Play It Again, Sam! An Analysis of Constructive Replication in the Organizational Sciences.” Journal of Management 47 (2): 488–518. https://doi.org/10.1177/0149206319843985.

Lakens, D. 2022. “Sample Size Justification.” Collabra: Psychology 8 (1): 33267. https://doi.org/10.1525/collabra.33267.

———. 2024. “When and How to Deviate from a Preregistration.” Collabra: Psychology 10 (1). https://doi.org/10.1525/collabra.117094.

Lakens, D., and A. J. Etz. 2017. “Too True to Be Bad: When Sets of Studies with Significant and Nonsignificant Findings Are Probably True.” Social Psychological and Personality Science 8 (8): 875–81. https://doi.org/10.1177/1948550617693058.

Landy, J. F., M. L. Jia, I. L. Ding, D. Viganola, W. Tierney, A. Dreber, and and Crowdsourcing Hypothesis Tests Collaboration. 2020. “Crowdsourcing Hypothesis Tests: Making Transparent How Design Choices Shape Research Results.” Psychological Bulletin 146 (5): 451. https://doi.org/10.1037/bul0000220.

Lash, T. L., L. J. Collin, and M. E. Van Dyke. 2018. “The Replication Crisis in Epidemiology: Snowball, Snow Job, or Winter Solstice?” Current Epidemiology Reports 5: 175–83.

LeBel, E. P., R. J. McCarthy, B. D. Earp, M. Elson, and W. Vanpaemel. 2018. “A Unified Framework to Quantify the Credibility of Scientific Findings.” Advances in Methods and Practices in Psychological Science 1 (3): 389–402. https://doi.org/10.1177/2515245918787489.

Mahoney, M. J. 1977. “Publication Prejudices: An Experimental Study of Confirmatory Bias in the Peer Review System.” Cognitive Therapy and Research 1: 161–75. https://doi.org/10.1007/BF01173636.

Makel, M. C., J. A. Plucker, and B. Hegarty. 2012. “Replications in Psychology Research: How Often Do They Really Occur?” Perspectives on Psychological Science 7 (6): 537–42. https://doi.org/10.1177/1745691612460688.

Mazei, J., J. Hüffmeier, and T. Schultze. 2025. “Specification Curve and Reproducibility Dashboards for Social Science Research: Recommendations for Implementation.” Advances in Methods and Practices in Psychological Science.

McManus, K. 2024. “Replication Studies in Second Language Acquisition Research: Definitions, Issues, Resources, and Future Directions: Introduction to the Special Issue.” Studies in Second Language Acquisition 46 (5): 1299–319. https://doi.org/10.1017/S0272263124000652.

Miłkowski, M., W. M. Hensel, and M. Hohol. 2018. “Replicability or Reproducibility? On the Replication Crisis in Computational Neuroscience and Sharing Only Relevant Detail.” Journal of Computational Neuroscience 45 (3): 163–72. https://doi.org/10.1007/s10827-018-0702-z.

Moreau, D., and K. Wiebels. 2023. “Ten Simple Rules for Designing and Conducting Undergraduate Replication Projects.” PLOS Computational Biology 19 (3): e1010957. https://doi.org/10.1371/journal.pcbi.1010957.

Munafò, M. R., C. D. Chambers, A. M. Collins, L. Fortunato, and M. R. Macleod. 2020. “Research Culture and Reproducibility.” Trends in Cognitive Sciences 24 (2): 91–93. https://doi.org/10.1016/j.tics.2019.12.002.

Nosek, B. A., and T. M. Errington. 2020. “What Is Replication?” PLoS Biology 18 (3): e3000691. https://doi.org/10.1371/journal.pbio.3000691.

Nuijten, M. B., and J. R. Polanin. 2020. “‘Statcheck’: Automatically Detect Statistical Reporting Inconsistencies to Increase Reproducibility of Meta‐analyses.” Research Synthesis Methods 11 (5): 574–79. https://doi.org/10.1002/jrsm.1408.

Nüst, D., and S. J. Eglen. 2021. “CODECHECK: An Open Science Initiative for the Independent Execution of Computations Underlying Research Articles During Peer Review to Improve Reproducibility.” F1000Research 10: 253. https://doi.org/10.12688/f1000research.51738.2.

Open Science Collaboration. 2015. “Estimating the Reproducibility of Psychological Science.” Science 349 (6251): aac4716. https://doi.org/10.1126/science.aac4716.

Patil, P., R. D. Peng, and J. T. Leek. 2016. “A Statistical Definition for Reproducibility and Replicability.” BioRxiv, 066803. https://doi.org/10.1101/066803.

Pennington, C. R. 2023. A Student’s Guide to Open Science: Using the Replication Crisis to Reform Psychology. Open University Press.

Perry, T., R. Morris, and R. Lea. 2022. “A Decade of Replication Study in Education? A Mapping Review (2011–2020).” Educational Research and Evaluation 27 (1-2): 12–34. https://doi.org/10.1080/13803611.2021.2022315.

Pittelkow, M. M., S. M. Field, P. M. Isager, A. E. van’t Veer, T. Anderson, S. N. Cole, and D. and Van Ravenzwaaij. 2023. “The Process of Replication Target Selection in Psychology: What to Consider?” Royal Society Open Science 10 (2): 210586. https://doi.org/10.1098/rsos.210586.

Pittelkow, M. M., R. Hoekstra, J. Karsten, and D. van Ravenzwaaij. 2021. “Replication Target Selection in Clinical Psychology: A Bayesian and Qualitative Reevaluation.” Clinical Psychology: Science and Practice 28 (2): 210. https://doi.org/10.1037/cps0000013.

Pittelkow, M., S. M. Field, and D. van Ravenzwaaij. 2025. “Thinking Beyond RVCN: Addressing the Complexity of Replication Target Selection.” https://doi.org/10.31234/osf.io/6tmyx_v2.

Pownall, M. 2022. “Is Replication Possible for Qualitative Research?” https://doi.org/10.31234/osf.io/dwxeg.

Protzko, J. 2018. “Null-Hacking, a Lurking Problem.” https://doi.org/10.31234/osf.io/9y3mp.

Röseler, L., L. Kaiser, C. Doetsch, N. Klett, C. Seida, A. Schütz, and Y. and Zhang. 2024. “The Replication Database: Documenting the Replicability of Psychological Science.” Journal of Open Psychology Data 12 (1): 8. https://doi.org/10.5334/jopd.101.

Schimmack, U. 2012. “The Ironic Effect of Significant Results on the Credibility of Multiple-Study Articles.” Psychological Methods 17 (4): 551. https://doi.org/10.1037/a0029487.

Schmidt, S. 2009. “Shall We Really Do It Again? The Powerful Concept of Replication Is Neglected in the Social Sciences.” Review of General Psychology 13 (2): 90–100. https://doi.org/10.1037/a0015108.

Schöch, C. 2023. “Repetitive Research: A Conceptual Space and Terminology of Replication, Reproduction, Revision, Reanalysis, Reinvestigation and Reuse in Digital Humanities.” International Journal of Digital Humanities 5 (2): 373–403. https://doi.org/10.1007/s42803-023-00073-y.

Simonsohn, U. 2015. “Small Telescopes: Detectability and the Evaluation of Replication Results.” Psychological Science 26 (5): 559–69. https://doi.org/10.1177/0956797614567341.

Simonsohn, U., J. P. Simmons, and L. D. Nelson. 2020. “Specification Curve Analysis.” Nature Human Behaviour 4: 1208–14. https://doi.org/10.1038/s41562-020-0912-z.

Soderberg, C. K., T. M. Errington, S. R. Schiavone, and and others. 2021. “Initial Evidence of Research Quality of Registered Reports Compared with the Standard Publishing Model.” Nature Human Behaviour 5: 990–97. https://doi.org/10.1038/s41562-021-01142-4.

Soto, C. J. 2019. “How Replicable Are Links Between Personality Traits and Consequential Life Outcomes? The Life Outcomes of Personality Replication Project.” Psychological Science 30 (5): 711–27. https://doi.org/10.1177/0956797619831612.

Syed, M. 2023. “Replication or Generalizability? How Flexible Inferences Uphold Unfounded Universal Claims,” May. https://doi.org/10.31234/osf.io/znv5r.

Tsang, E. W., and K. M. Kwan. 1999. “Replication and Theory Development in Organizational Science: A Critical Realist Perspective.” Academy of Management Review 24 (4): 759–80. https://doi.org/10.2307/259353.

Urminsky, O., and B. J. Dietvorst. 2024. “Taking the Full Measure: Integrating Replication into Research Practice to Assess Generalizability.” Journal of Consumer Research 51 (1): 157–68. https://doi.org/10.1093/jcr/ucae007.

Voelkl, B., R. Heyard, D. Fanelli, K. E. Wever, L. Held, Z. Maniadis, and H. and Würbel. 2025. “Defining Reproducibility.” https://doi.org/10.17605/OSF.IO/BR9SP.

Vohs, K. D., B. J. Schmeichel, S. Lohmann, Q. F. Gronau, A. J. Finley, S. E. Ainsworth, J. L. Alquist, et al. 2021. “A Multisite Preregistered Paradigmatic Test of the Ego-Depletion Effect.” Psychological Science 32 (10): 1566–81. https://doi.org/10.1177/0956797621989733.

Wilkinson, M. D., M. Dumontier, I. J. Aalbersberg, G. Appleton, M. Axton, A. Baak, and B. and Mons. 2016. “The FAIR Guiding Principles for Scientific Data Management and Stewardship.” Scientific Data 3 (1): 1–9. https://doi.org/10.1038/sdata.2016.18.

Willroth, E. C., and O. E. Atherton. 2024. “Best Laid Plans: A Guide to Reporting Preregistration Deviations.” Advances in Methods and Practices in Psychological Science 7 (1): 25152459231213802. https://doi.org/10.1177/25152459231213802.

Yarkoni, T. 2013. “‘What We Can and Can’t Learn from the Many Labs Replication Project’.” Talyarkoni.org/Blog.

Zwaan, R. A., A. Etz, R. E. Lucas, and M. B. Donnellan. 2018. “Making Replication Mainstream.” Behavioral and Brain Sciences 41: e120. https://doi.org/10.1017/S0140525X17001972.