Explore replication and reproduction evidence
Find original reports, attempts to repeat them, and the evidence behind each recorded outcome. Each row is a reference pair: one original report and one repetition report. A report can occur in several pairs. To look up studies, use the Replication Atlas.
Citation and reuse guidance
The projects embedded in this website are the work of over two hundred paid and volunteer researchers over more than four years. Please cite the respective works when (re-)using them. If you expect to profit from this work, for example through a grant application based on these data, please consider reaching out to us to involve members of the community.
Outcome Breakdown
About FLoRA
What is the aim of this project?
FLoRA tracks numerical reproductions, robustness reproductions, and replications, linking each one to the original publication it tests. The goal is to increase the findability of repetition attempts and to enable meta-research on the repeatability of scientific findings, across disciplines, journals, and time.
FLoRA tracks attempts to repeatedly test published findings across the sciences and humanities. Unlike the FORRT Replication Database (FReD), FLoRA does not include statistical data. This lets it cover every field of research and allows much faster expansion.
What is a replication?
Replications intentionally repeat prior research to test whether the original findings hold. To be included in FLoRA, a study must:
- Self-identify as a replication before reporting results: replication must be an aim, not just a result
- Identify a specific target study (or studies)
- Replicate a study or experiment: a central claim, not just any empirical pattern
Replications range from close/direct (same methods, same population) to conceptual (same hypothesis, different methods). Outcomes are tagged Successful, Failed, or Mixed based on how the replication authors characterise their results.
What is a reproduction?
Reproductions verify whether reported results can be obtained from the original study's data and methods. They are coded on two dimensions:
- Computational success: were the original results obtained? (Computationally Successful, Computational Issues, or Technical Failure)
- Robustness: do results hold under reasonable alternative specifications? (Robust or Robustness Challenges)
The two dimensions are assessed independently, and not every reproduction is assessed on both. Charts for each dimension count only the studies that carry an actual verdict on it, so a reproduction whose robustness was never checked still appears under computational reproducibility.
Structure of FLoRA
FLoRA links each repetition reference to its original, and assigns a standardized outcome based on the repetition report. This simple structure lets us track replication rates across time, disciplines, and journals, and powers tools like the FLoRA Annotator, which lists replication and reproduction attempts for any reference you provide.
Is FLoRA representative and comprehensive?
Honestly — neither. We are still working on a systematic literature search, and several structural factors limit coverage:
- Some replications cannot be added because they do not self-identify as such in the metadata we can reach (e.g., the title and abstract).
- In some areas of computational reproduction (for example, CODECHECK certificates), the artefacts are typically only issued for successful numerical reproductions, which makes representative estimates of computational reproducibility difficult.
- Coverage by discipline, language, and publication venue is uneven and reflects where contributors have focused so far.
Treat aggregate statistics as exploratory rather than definitive, and please flag missing entries when you spot them.
FLoRA is the product of years of work by a large community of volunteers. If you'd like to contribute or use the data for your own research, please reach out.
FAQs
If anything is unclear, here is where we collect frequently asked questions. The list is loaded live from the FLoRA FAQ document and grows as new questions come in.
More Replication Tools
Recommended Citation
If you use the underlying dataset, please cite it as:
If you report results based on this website, please cite it as:
Citation and reuse guidance
The projects embedded in this website are the work of over two hundred paid and volunteer researchers over more than four years. Please cite the respective works when (re-)using them. If you expect to profit from this work, for example through a grant application based on these data, please consider reaching out to us to involve members of the community.
Outcome Breakdown
Studies Database
| Original Study | Year | Replication Study | Year | Outcome | Type | Original DOI | Replication Report |
|---|
Citation and reuse guidance
Outcomes by original study year
How have the findings of studies from different eras fared when scientists have tried to replicate them? Each bar groups replication and reproduction attempts by the year the original study was published, split into Successful, Mixed, Failed, and other outcomes.
Use this to see how findings from different historical periods have held up in subsequent replication efforts.
Studies without a recorded original publication year are excluded.
Outcomes by replication study year
Grouped by the year of the replication itself, this view shows the growth of FLoRA's coverage and how outcome profiles have shifted over time as replication practice has matured.
The rightmost years may reflect ongoing data entry rather than a genuine decline in activity.
Replications without a recorded year are excluded.
Outcomes by original journal
The journals whose articles have most frequently been the target of replication attempts. For each journal, the bar shows the outcome breakdown of those attempts.
Frequency here reflects where replication attention has been focused, not necessarily a journal's overall replicability.
Outcomes by replication journal
Where replication studies are themselves being published. Each bar shows the outcome breakdown of replications appearing in a given journal.
This view highlights the venues that publish replication research most actively, including journals dedicated to replications and registered reports.
Annotation of replication-journal data is a work in progress; some journals may be missing or under-represented at the moment.
Outcomes by field
A bird's-eye view grouping studies by research discipline. Each bar shows the outcome breakdown of replication and reproduction attempts targeting work originally published in journals from that field.
Disciplines are inferred from the original study's journal name using a hand-curated dictionary (data/disciplines.json). Journals that have not been mapped yet are collected under Uncategorized.
Coverage is still growing. Many journals are not yet classified, so some fields may be under-represented and Uncategorized may dominate. The dictionary is straightforward to extend.
Citation and reuse guidance
This is a simplified preview of the methods pre-registered for our ongoing confirmatory study:
Citation patterns around the first replication
This tab combines the FLoRA dataset with citation data from OpenCitations to describe citations to the original around the first recorded replication, and how often the two are "co-cited" (cited together by the same later work).
Reproduction citation counts
Counts are grouped by the assessed dimension. A report can contribute to several outcome groups, so group totals are not additive.
| Originals | Citations to original | Citations to reproduction | Co-citations |
|---|
Aggregate impact of replications
Event-study estimates: each original is aligned at t = 0, the year of its first published replication. Raw trajectories show average counts. The separate coefficient plots show OLS estimates and 95% confidence intervals on the log(1 + count) scale, relative to year −1, adjusting for original and calendar-year fixed effects. These associations do not establish a causal effect.
Citations to the original
Co-citations (original and replication)
Co-citation rate by replication report size & outcome
The co-citation rate is the share of an original's citations that also cite one of its replications, counted from the year of its first replication. Originals with multiple replications appear in every row that applies. Rates vary widely across originals, so three summaries are shown: per-paper mean (each original weighted equally), median (robust to outliers), and weighted mean (pooled co-citations ÷ pooled citations, effectively weighted by citation volume).
Browse all originals
Click any row to see its citation timeline with replication events marked.
| Original | Year | Replications | Citations | Co-citations |
|---|
About this analysis
Data are refreshed weekly. The pipeline filters FLoRA to conceptual and direct replications with outcomes labelled successful, failed, or mixed (reproductions are excluded). For each included original and replication, we retrieve all citing works from OpenCitations COCI. A "co-citation" is any work that cites both the original and at least one of its replications.
The outcome chips group originals by the outcome of their earliest recorded replication year; same-year ties follow the stored record order. Later replications can have different outcomes. The table’s outcome filter instead matches any replication of an original. Qualified success, descriptive-only, uninformative, and missing outcomes are excluded from the citation model.
Each timeline bar counts distinct citing works. Works that cite replications with more than one outcome appear once in the “multiple outcome categories” segment. The outcome-specific co-citation rate tables are overlapping summaries and should not be added.
The aggregate plots show event-study estimates from an OLS two-way fixed-effects model, a transparent analogue of the FECTFixed Effects Counterfactual estimator: the causal-inference method specified in our preregistered protocol. This chart approximates it with a simpler OLS two-way fixed-effects model. specification used in the preregistered analysis. See the FReD-data repository for the underlying FLoRA dataset.
Citation and reuse guidance
Journal-Level Citation Metrics and Replicability
This tab presents a statistical analysis of whether the OpenAlex Mean Citedness (OMC) of an original study's journal predicts replication success. OMC is a proxy for the Journal Impact Factor. Mean Citedness is a journal-level measure from OpenAlex: the mean citations to its works published in the previous two years. It is retrieved at enrichment time, not matched to the original publication year, and is not the original article’s citation count. The update stamp identifies the analysis snapshot; the model is a descriptive association.
Distribution of studies by journal Mean Citedness
P(successful replication) vs Mean Citedness — logistic smooth
The model conditions on unqualified successful or failed outcomes. Other outcomes, including qualified success, are excluded. A low p-value does not imply strong prediction.
Citation and reuse guidance
Authorship Overlap and Outcomes
This analysis checks whether any author of the original study also appears as an author of the replication study: a pattern sometimes called a same-author replication or self-replication. Author names are matched by family name (normalised: lowercase, diacritics removed). This may produce false positives for common family names when two unrelated researchers share a surname.
Replication outcomes by authorship overlap
Citation and reuse guidance
Recorded publication venue
Venue names identify repositories/preprints, conference outputs, and theses where recognisable. Missing metadata remain unknown. All other names are labelled “other named venue”; this does not verify a journal article or peer review. Percentages use all reference pairs in each venue group.
Outcomes by publication venue
journal_r field recorded in FLoRA, classified as above. Some studies
currently classified as preprints may since have been formally published in a
different venue without FLoRA yet reflecting that update. Any outcome
difference between the two groups should also not be read as caused by publication
venue alone: constructive peer review, a tendency for more selective journals to
accept more polished work, and publication-pressure effects — for example a bias
toward reporting positive or successful results in order to get published, or
conversely extra editorial scrutiny catching more issues before publication — could
all plausibly produce real differences independent of, or entangled with, venue
itself.
All studies with publication-status data
| Study | Journal | Status | Year | Outcome | DOI |
|---|
Publication Format: Registered Reports vs. Other Replications
This analysis checks whether each replication in FLoRA was itself published as a Registered Report (RR): a format in which the study design and analysis plan are peer-reviewed and provisionally accepted before results are known. Matching is done by DOI/title against the Registered Reports Database (RRDB), a regularly-updated export of the FORRT Zotero "Registered Reports" group library. Replications found there go into the Registered Report group; every other replication falls into the rest group.
Because RRDB is itself a work in progress, "rest" really means not (yet) found in the database, rather than confirmed non-RR.
Citation and reuse guidance
Replication outcomes by Registered Report status
Replications identified as Registered Reports
| Replication Study | Journal | Year | Outcome | DOI |
|---|
Reports by number of original targets
This analysis groups replication and reproduction reports by their number of recorded original targets. Multiple-target reports contain more than three distinct original targets in the recorded data. The remaining group contains reports with up to three targets and records with missing report identifiers. Target count does not establish the number of laboratories or membership of a coordinated project. Chart percentages count reference pairs; the evidence list counts reports.
Outcomes by report size
Multiple-target reports
| Project | Journal | Year | Targets | Outcome mix | DOI |
|---|