Multiverse analysis

Definition: Multiverse analyses are based on all potentially equally justifiable data processing and statistical analysis pipelines that can be employed to test a single hypothesis. In a data multiverse analysis, a single set of raw data is processed into a multiverse of data sets by applying all possible combinations of justifiable preprocessing choices. Model multiverse analyses apply equally justifiable statistical models to the same data to answer the same hypothesis. The statistical analysis is then conducted on all data sets in the multiverse and all results are reported which enhances promoting transparency and illustrates the robustness of results against different data processing (data multiverse) or statistical (model multiverse) pipelines). Multiverse analysis differs from Specification curve analysis with regards to the graphical displays (a histogram and tile plota rather than a specification curve plot).

Related terms: Garden of forking paths, Robustness (analyses), Specification curve analysis, Vibration of effects

References: Del Giudice and Gangestad (2021), & Steegen et al. (2016)

Drafted and Reviewed by: Tina Lonsdorf, Flávio Azevedo, Mahmoud Elsherif, Adrien Fillon, William Ngiam, Sam Parsons

Note that we are currently working on an automated mechanism to link references cited above with their full-length version that can be found at https://forrt.org/glossary/references with all references used so far.