Multi-Analyst Studies
Definition: In typical empirical studies, a single researcher or research team conducts the analysis, which creates uncertainty about the extent to which the choice of analysis influences the results. In multi-analyst studies, two or more researchers independently analyse the same research question or hypothesis on the same dataset. According to Aczel and colleagues (2021), a multi-analyst approach may be beneficial in increasing our confidence in a particular finding; uncovering the impact of analytical preferences across research teams; and highlighting the variability in such analytical approaches.
Related terms: Analytic flexibility, Crowdsourcing science, Data Analysis, Garden of Forking Paths, Multiverse Analysis, Researcher Degrees of Freedom, Scientific Transparency
References:
- Aczel, B., Szaszi, B., Nilsonne, G., Van den Akker, O., Albers, C. J., van Assen, M. A. L. M., …, & Wagenmakers, E. (2021). Guidance for Multi-Analyst Studies. https://doi.org/10.31222/osf.io/5ecnh
- Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E., BahnĂk, Ć ., Bai, F., Bannard, C., Bonnier, E., & others. (2018). Many Analysts, One Data Set: Making Transparent How Variations in Analytic Choices Affect Results. Advances in Methods and Practices in Psychological Science, 337â356. https://doi.org/10.1177/2515245917747646
Originally drafted by: Sam Parsons
Reviewed by: Tsvetomira Dumbalska, Mahmoud Elsherif, William Ngiam, Charlotte R. Pennington, Graham Reid, Barnabas Szaszi, FlĂĄvio Azevedo