The projects embedded in this website are the work of over two hundred paid and volunteer researchers over more than four years. Please cite the respective works when (re-)using them. If you expect to profit from this work, for example through a grant application based on these data, please consider reaching out to us to involve members of the community.
What is the aim of this project?
FLoRA tracks numerical reproductions, robustness reproductions, and replications, linking each one to the original publication it tests. The goal is to increase the findability of repetition attempts and to enable meta-research on the repeatability of scientific findings, across disciplines, journals, and time.
FLoRA tracks attempts to repeatedly test published findings across the sciences and humanities. Unlike the FORRT Replication Database (FReD), FLoRA does not include statistical data. This lets it cover every field of research and allows much faster expansion.
What is a replication?
Replications intentionally repeat prior research to test whether the original findings hold. To be included in FLoRA, a study must:
- Self-identify as a replication before reporting results: replication must be an aim, not just a result
- Identify a specific target study (or studies)
- Replicate a study or experiment: a central claim, not just any empirical pattern
Replications range from close/direct (same methods, same population) to conceptual (same hypothesis, different methods). Outcomes are tagged Successful, Failed, or Mixed based on how the replication authors characterise their results.
What is a reproduction?
Reproductions verify whether reported results can be obtained from the original study's data and methods. They are coded on two dimensions:
- Computational success: were the original results obtained? (Computationally Successful vs Computational Issues)
- Robustness: do results hold under reasonable alternative specifications? (Robust, Robustness Challenges, or Robustness Not Checked)
Structure of FLoRA
FLoRA links each repetition reference to its original, and assigns a standardized outcome based on the repetition report. This simple structure lets us track replication rates across time, disciplines, and journals, and powers tools like the FLoRA Annotator, which lists replication and reproduction attempts for any reference you provide.
Is FLoRA representative and comprehensive?
Honestly — neither. We are still working on a systematic literature search, and several structural factors limit coverage:
- Some replications cannot be added because they do not self-identify as such in the metadata we can reach (e.g., the title and abstract).
- In some areas of computational reproduction (for example, CODECHECK certificates), the artefacts are typically only issued for successful numerical reproductions, which makes representative estimates of computational reproducibility difficult.
- Coverage by discipline, language, and publication venue is uneven and reflects where contributors have focused so far.
Treat aggregate statistics as exploratory rather than definitive, and please flag missing entries when you spot them.
FLoRA is the product of years of work by a large community of volunteers. If you'd like to contribute or use the data for your own research, please reach out.
If anything is unclear, here is where we collect frequently asked questions. The list is loaded live from the FLoRA FAQ document and grows as new questions come in.
If you use the underlying dataset, please cite it as:
If you report results based on this website, please cite it as:
The projects embedded in this website are the work of over two hundred paid and volunteer researchers over more than four years. Please cite the respective works when (re-)using them. If you expect to profit from this work, for example through a grant application based on these data, please consider reaching out to us to involve members of the community.
| Original Study | Year | Replication Study | Year | Outcome | Type | Original DOI | Replication Report |
|---|
Outcomes by original study year
How have the findings of studies from different eras fared when scientists have tried to replicate them? Each bar groups replication and reproduction attempts by the year the original study was published, split into Successful, Mixed, Failed, and other outcomes.
Use this to see how findings from different historical periods have held up in subsequent replication efforts.
Studies without a recorded original publication year are excluded.
Outcomes by replication study year Experimental · WIP
Grouped by the year of the replication itself, this view shows the growth of FLoRA's coverage and how outcome profiles have shifted over time as replication practice has matured.
The rightmost years may reflect ongoing data entry rather than a genuine decline in activity.
Replications without a recorded year are excluded.
Outcomes by original journal
The journals whose articles have most frequently been the target of replication attempts. For each journal, the bar shows the outcome breakdown of those attempts.
Frequency here reflects where replication attention has been focused, not necessarily a journal's overall replicability.
Outcomes by replication journal Experimental · WIP
Where replication studies are themselves being published. Each bar shows the outcome breakdown of replications appearing in a given journal.
This view highlights the venues that publish replication research most actively, including journals dedicated to replications and registered reports.
Annotation of replication-journal data is a work in progress; some journals may be missing or under-represented at the moment.
Outcomes by field Experimental · WIP
A bird's-eye view grouping studies by research discipline. Each bar shows the outcome breakdown of replication and reproduction attempts targeting work originally published in journals from that field.
Disciplines are inferred from the original study's journal name using a hand-curated dictionary (data/disciplines.json). Journals that have not been mapped yet are collected under Uncategorized.
Coverage is still growing. Many journals are not yet classified, so some fields may be under-represented and Uncategorized may dominate. The dictionary is straightforward to extend.
This is a simplified preview of the methods pre-registered for our ongoing confirmatory study:
How do replications change citation behaviour?
Replications only: reproductions are not included in this analysisThis tab combines the FLoRA dataset with citation data from OpenCitations to explore how the publication of a replication affects subsequent citations to the original study, and how often the two are "co-cited" (cited together by the same later work).
Event-study estimates: each original is aligned at t = 0, the year of its first published replication. Bands show 95% CIs from an OLS model on log(1 + citations) with study and year fixed effects (a two-way fixed-effects, or TWFE, analogue of the preregistered FECTFixed Effects Counterfactual estimator: the causal-inference method specified in our preregistered protocol. This chart approximates it with a simpler OLS two-way fixed-effects model. model).
Citations to the original
Co-citations (original ⊕ replication)
The co-citation rate is the share of an original's citations that also cite one of its replications, counted from the year of its first replication. Originals with multiple replications appear in every row that applies. Rates vary widely across originals, so three summaries are shown: per-paper mean (each original weighted equally), median (robust to outliers), and weighted mean (pooled co-citations ÷ pooled citations, effectively weighted by citation volume).
Click any row to see its citation timeline with replication events marked.
| Original | Year | Replications | Citations | Co-citations |
|---|
Data are refreshed weekly. The pipeline filters FLoRA to conceptual and direct replications with outcomes labelled successful, failed, or mixed (reproductions are excluded). For each included original and replication, we retrieve all citing works from OpenCitations COCI. A "co-citation" is any work that cites both the original and at least one of its replications.
The aggregate plots show event-study estimates from an OLS two-way fixed-effects model, a transparent analogue of the FECTFixed Effects Counterfactual estimator: the causal-inference method specified in our preregistered protocol. This chart approximates it with a simpler OLS two-way fixed-effects model. specification used in the preregistered analysis. See the FReD-data repository for the underlying FLoRA dataset.
This tab presents a statistical analysis of whether the OpenAlex Mean Citedness (OMC) of an original study's journal predicts replication success. OMC is a proxy for the Journal Impact Factor. The analysis is rendered weekly by an R pipeline (GAM).
Replications only: reproductions are not included in this analysisOnly studies with successful or failed replication outcomes were included in the smooth model. Mixed and inconclusive outcomes were excluded.
This analysis checks whether any author of the original study also appears as an author of the replication study: a pattern sometimes called a same-author replication or self-replication. Author names are matched by family name (normalised: lowercase, diacritics removed). This may produce false positives for common family names when two unrelated researchers share a surname.
Replications only: reproductions are not included in this analysis