Replication Research

 

The Generalizability of Survey Experiments

Survey experiments have become a central methodology across the social sciences. Researchers can combine experiments’ causal power with the generalizability of population-based samples. Yet, due to the expense of population-based samples, much …

The GRIM Test: A Simple Technique Detects Numerous Anomalies in the Reporting of Results in Psychology

We present a simple mathematical technique that we call granularity-related inconsistency of means (GRIM) for verifying the summary statistics of research reports in psychology. This technique evaluates whether the reported means of integer data such …

The Hardest Science

Blogposts about psychology, reproducibility, replication etc.

The logical structure of experiments lays the foundation for a theory of reproducibility

The scientific reform movement has proposed openness as a potential remedy to the putative reproducibility or replication crisis. However, the conceptual relationship among openness, replication experiments and results reproducibility has been …

The meaning of “significance” for different types of research

Adrianus Dingeman de Groot (1914-2006) was one of the most influential Dutch psychologists. He became famous for his work "Thought and Choice in Chess", but his main contribution was methodological--De Groot co-founded the Department of Psychological …

The natural selection of bad science.

Poor research design and data analysis encourage false-positive findings. Such poor methods persist despite perennial calls for improvement, suggesting that they result from something more than just misunderstanding. The persistence of poor methods …

The need for public opinion and survey methodology research to embrace preregistration and replication, exemplified by a team’s failure to replicate their own findings on visual cues in grid-type questions

Survey researchers take great care to measure respondents’ answers in an unbiased way; but, how successful are we as a field at remedying unintended and intended biases in our research? The validity of inferences drawn from studies has been found to …

The Null Hypothesis Significance-Testing Debate and Its Implications for Personality Research

A chapter about null hypothesis significance testing in personality research

The Persistence of Underpowered Studies in Psychological Research: Causes, Consequences, and Remedies

Underpowered studies persist in the psychological literature. This article examines reasons for their persistence and the effects on efforts to create a cumulative science. The "curse of multiplicities" plays a central role in the presentation. Most …

The pipeline project: Pre-publication independent replications of a single laboratory's research pipeline

This crowdsourced project introduces a collaborative approach to improving the reproducibility of scientific research, in which findings are replicated in qualified independent laboratories before (rather than after) they are published. Our goal is …
JUST-OS