Many Labs
Definition: A series of crowdsourced replication projects in which independent research teams use shared study protocols to examine the replicability of published psychological effects and variation in results across samples and settings. The first project, Many Labs 1 (Klein et al., 2014), tested 13 effects across 36 independent samples involving 6,344 participants. It was followed by Many Labs 2 (Klein et al., 2018) and further projects investigating different facets of replicability. Similar collaborative initiatives include ManyBabies, EEGManyLabs, and the Psychological Science Accelerator.
Related terms: Collaboration, Many analysts, Many Labs I, Many Labs II, Open Science Collaboration, Replication
References:
- Ebersole, C. R., Atherton, O. E., Belanger, A. L., Skulborstad, H. M., Allen, J. M., Banks, J. B., & Nosek, B. A. (2016). Many Labs 3: Evaluating participant pool quality across the academic semester via replication. Journal of Experimental Social Psychology, 67, 68–82. https://doi.org/10.1016/j.jesp.2015.10.012
- Frank, M. C., Bergelson, E., Bergmann, C., Cristia, A., Floccia, C., Gervain, J., Hamlin, J. K., Hannon, E. E., Kline, M., Levelt, C., Lew-Williams, C., Nazzi, T., Panneton, R., Rabagliati, H., Soderstrom, M., Sullivan, J., Waxman, S., & Yurovsky, D. (2017). A Collaborative Approach to Infant Research: Promoting Reproducibility, Best Practices, and Theory-Building. Infancy, 22, 421–435. https://doi.org/10.1111/infa.12182
- Klein, R. A., Ratliff, K. A., Vianello, M., Adams, R. B., BahnĂk, Ĺ ., Bernstein, M. J., & et al. (2014). Investigating variation in replicability: A “many labs” replication project. Social Psychology, 45, 142–152. https://doi.org/10.1027/1864-9335/a000178
- Klein, R. A., Vianello, M., Hasselman, F., Adams, B. G., Adams, R. B., Alper, S., & … Nosek, B. A. (2018). Many Labs 2: Investigating Variation in Replicability Across Samples and Settings. Advances in Methods and Practices in Psychological Science, 1(4), 443–490. https://doi.org/10.1177/2515245918810225
- Moshontz, H., Campbell, L., Ebersole, C. R., IJzerman, H., Urry, H. L., Forscher, P. S., & Chartier, C. R. (2018). The Psychological Science Accelerator: Advancing psychology through a distributed collaborative network. Advances in Methods and Practices in Psychological Science, 1(4), 501–515. https://doi.org/10.1177/2515245918797607
- Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. https://doi.org/10.1126/science.aac4716
- Pavlov, Y. G., Adamian, N., Appelhoff, S., Arvaneh, M., Benwell, C., Beste, C., & Mushtaq, F. (2020). #EEGManyLabs: Investigating the Replicability of Influential EEG Experiments. PsyArXiv Preprint. https://doi.org/10.31234/osf.io/528nr
Originally drafted by: Sam Parsons
Reviewed by: Helena Hartmann, Charlotte R. Pennington, Mirela Zaneva
Suggest an improvement
Spotted an error, a broken link, or think this entry is missing something? Let us know — this goes directly to the glossary team.