Replication Research

‘‘Positive’’ Results Increase Down the Hierarchy of the Sciences

The hypothesis of a Hierarchy of the Sciences with physical sciences at the top, social sciences at the bottom, and biological sciences in-between is nearly 200 years old. This order is intuitive and reflected in many features of academic life, but …

1Economics 270D: Research Transparency Methods in the Social Sciences

This course coversa range of approachesthat aim to enhance the transparency and reproducibility of social science research. It is appropriate for Ph.D. students in social science disciplines and related fields.

A 21 word solution.

One year after publishing "False-Positive Psychology," we propose a simple implementation of disclosure that requires but 21 words to achieve full transparency. This article is written in a casual tone. It includes phone-taken pictures of milk-jars …

A Bayesian Perspective on the Reproducibility Project: Psychology

We revisit the results of the recent Reproducibility Project: Psychology by the Open Science Collaboration. We compute Bayes factors—a quantity that can be used to express comparative evidence for an hypothesis but also for the null hypothesis—for a …

A collaborative approach to infant research: Promoting reproducibility, best practices, and theory-building.

The ideal of scientific progress is that we accumulate measurements and integrate these into theory, but recent discussion of replicability issues has cast doubt on whether psychological research conforms to this model. Developmental …

A Duty to Describe: Better the Devil You Know Than the Devil You Don't

Although many researchers have discussed replication as a means to facilitate self-correcting science, in this article, we identify meta-analyses and evaluating the validity of correlational and causal inferences as additional processes crucial to …

A manifesto for reproducible science

Improving the reliability and efficiency of scientific research will increase the credibility of the published scientific literature and accelerate discovery. Here we argue for the adoption of measures to optimize key elements of the scientific …

A multilab preregistered replication of the ego-depletion effect.

Good self-control has been linked to adaptive outcomes such as better health, cohesive personal relationships, success in the workplace and at school, and less susceptibility to crime and addictions. In contrast, self-control failure is linked to …

A Powerful Nudge? Presenting Calculable Consequences of Underpowered Research Shifts Incentives Toward Adequately Powered Designs

If psychologists have recognized the pitfalls of underpowered research for decades, why does it persist? Incentives, perhaps: underpowered research benefits researchers individually (increased productivity), but harms science collectively (inflated …

A Reliability-Generalization Study of Journal Peer Reviews: A Multilevel Meta-Analysis of Inter-Rater Reliability and Its Determinants

Background: This paper presents the first meta-analysis for the inter-rater reliability (IRR) of journal peer reviews. IRR is defined as the extent to which two or more independent reviews of the same scientific document agree. Methodology/Principal …