Robustness (analyses)

Related terms: Many Labs, Multiverse analysis, Sensitivity analyses, Specification Curve Analysis Alternative definition: “Robustness refers to the stability of experimental conclusions to variations in either baseline assumptions or experimental procedures. It is somewhat related to the concept of generalizability (also known as transportability), which refers to the persistence of an effect in settings different from and outside of an experimental framework [...] Whether a study design is similar enough to the original to be considered a replication, a “robustness test,” or some of many variations of pure replication that have been identified, particularly in the social sciences (for example, conceptual replication, pseudoreplication), is an unsettled question” (Goodman et al., 2016). Alternative definition: “Robustness refers to the stability of experimental conclusions to variations in either baseline assumptions or experimental procedures. It is somewhat related to the concept of generalizability (also known as transportability), which refers to the persistence of an effect in settings different from and outside of an experimental framework [...] Whether a study design is similar enough to the original to be considered a replication, a “robustness test,” or some of many variations of pure replication that have been identified, particularly in the social sciences (for example, conceptual replication, pseudoreplication), is an unsettled question” (Goodman et al., 2016).

Reference: Goodman et al. (2016) (alternative); Nosek and Errington (2020)

Drafted and Reviewed by: Tina Lonsdorf; Flávio Azevedo, Gilad Feldman, Adrien Fillon, Helena Hartmann, Timo Roettger