1  Background

Note — Preliminary Version 0.1

This is a preliminary version. Feedback welcome: lukas.roeseler@uni-muenster.de or GitHub.

“The proof established by the test must have a specific form, namely, repeatability. The issue of the experiment must be a statement of the hypothesis, the conditions of test, and the results, in such form that another experimenter, from the description alone, may be able to repeat the experiment. Nothing is accepted as proof, in psychology or in any other science, which does not conform to this requirement.” – (Dunlap, 1926)

Repeatability is the cornerstone of many sciences: A majority of the scientific progress rests on the successful accumulation of evidence for claims through reproduction and replications to establish robust discoveries. Reproductions and replications, that is repeated testing of a hypothesis with the same (reproduction) or different (replication) data, are necessary.

Cumulative science without repetition is costly. The aim of this guide is to empower researchers to conduct high-quality reproductions and replications and thereby contribute to making their fields of research more cumulative and robust. Issues of replicability have been discussed across many disciplines, such as psychology (Open Science Collaboration, 2015), economics (Dreber & Johannesson, 2024), biology (Errington et al., 2021), marketing (Urminsky & Dietvorst, 2024), linguistics (McManus, 2024), computer science (Hummel & Manner, 2024) and epidemiology (Lash et al., 2018) and the number of replications has been rising sharply (see Figure 1.1).

Figure 1.1: Number of replication studies by year of publication, based on the FORRT Replication Database as of July 16, 2025.

While the number of replication and reproduction studies has increased, the overall proportion of them is still very small, with reviews finding yearly replication rates of up to 1% (Perry et al., 2022). Moreover, much of the guidance on replications is being developed actively (Clarke et al., 2024) and in narrow parts of science, which leads to fragmentation, siloing, and potentially inconsistent information.

Here we attempt to integrate useful guidelines (e.g., Block & Kuckertz, 2018; Jekel et al., 2020) into a comprehensive overview that allows diverse fields to profit from each other. In sum, this guide provides information about the entire process of research allowing researchers at all career stages to plan, conduct, and publish reproduction and replication studies. We limit our scope to quantitative research, given that the concept of reproducibility and replicability itself is highly contested among qualitative researchers (Bennett, 2021; Cole et al., 2024; see Makel et al., 2012; Pownall, 2022).