Robustness (analyses)
Definition: The persistence of support for a hypothesis under perturbations of the methodological/analytical pipeline In other words, applying different methods/analysis pipelines to examine if the same conclusion is supported under analytical different conditions.
Alternative definition: âRobustness refers to the stability of experimental conclusions to variations in either baseline assumptions or experimental procedures. It is somewhat related to the concept of generalizability (also known as transportability), which refers to the persistence of an effect in settings different from and outside of an experimental framework […] Whether a study design is similar enough to the original to be considered a replication, a ârobustness test,â or some of many variations of pure replication that have been identified, particularly in the social sciences (for example, conceptual replication, pseudoreplication), is an unsettled questionâ (Goodman et al., 2016).
Related terms: Many Labs, Multiverse analysis, Sensitivity analyses, Specification Curve Analysis
References: Goodman et al. (2016) (alternative), & Nosek and Errington (2020)
Drafted and Reviewed by: Tina Lonsdorf, FlĂĄvio Azevedo, Gilad Feldman, Adrien Fillon, Helena Hartmann, Timo Roettger