Multi-Analyst Studies

Definition: In typical empirical studies, a single researcher or research team conducts the analysis, which creates uncertainty about the extent to which the choice of analysis influences the results. In multi-analyst studies, two or more researchers independently analyse the same research question or hypothesis on the same dataset. According to Aczel and colleagues (2021), a multi-analyst approach may be beneficial in increasing our confidence in a particular finding; uncovering the impact of analytical preferences across research teams; and highlighting the variability in such analytical approaches.

Related terms: Analytic flexibility, Crowdsourcing science, Data Analysis, Garden of Forking Paths, Multiverse Analysis, Researcher Degrees of Freedom, Scientific Transparency

References: Aczel et. al. (2021), & Silberzahn et al. (2018)

Drafted and Reviewed by: Sam Parsons, Tsvetomira Dumbalska, Mahmoud Elsherif, William Ngiam, Charlotte R. Pennington, Graham Reid, Barnabas Szaszi, FlĂĄvio Azevedo

Note that we are currently working on an automated mechanism to link references cited above with their full-length version that can be found at https://forrt.org/glossary/references with all references used so far.