Resources

 

We have an amazing team who curated many resources for the community.

Preregistration of Epidemiologic Studies: An Ill-founded Mix of Ideas

The idea for the registration of observational epidemiologic studies (with their protocols, hypotheses, and analysis plans) is modeled on the registration of randomized clinical trials (RCTs). The movement that led to the registration of RCTs was fed …

Preregistration of Modeling Exercises May Not Be Useful

This is a commentary on Lee et al.’s (2019) article encouraging preregistration of model development, fitting, and evaluation. While we are in general agreement with Lee et al.’s characterization of the modeling process, we disagree on whether …

Preregistration of Study Protocols Is Unlikely to Improve the Yield From Our Science, But Other Strategies Might

A recent workshop focused on the idea that protocols for nonrandomized studies should be registered in advance of their conduct. In reviewing the workshop report and publications citing it, I note a number of unfounded assumptions and little …

Replication is Relevant in Qualitative Research

Replication has received increasing attention over the last decade. This comes on the heels of prominent instances of data fabrication (National Academies of Sciences, Engineering, and Medicine, 2017) and estimates that few studies attempt to …

Rethinking transparency and rigor from a qualitative open science perspective

Discussions around transparency in open science focus primarily on sharing data, materials, and coding schemes, especially as these practices relate to reproducibility. This fairly quantitative perspective of transparency does not align with all …

Robust Modeling in Cognitive Science

In an attempt to increase the reliability of empirical findings, psychological scientists have recently proposed a number of changes in the practice of experimental psychology. Most current reform efforts have focused on the analysis of data and the …

Should Preregistration of Epidemiologic Study Protocols Become Compulsory? Reflections and a Counterproposal

There is an ongoing debate regarding preregistration of epidemiologic study protocols. We examine the basic idea that preregistration of study protocols and their associated hypotheses would enhance the reliability of observational research. We …

The case for formal methodology in scientific reform

Current attempts at methodological reform in sciences come in response to an overall lack of rigor in methodological and scientific practices in experimental sciences. However, most methodological reform attempts suffer from similar mistakes and …

The Costs of HARKing

Kerr ([1998]) coined the term ‘HARKing’ to refer to the practice of ‘hypothesizing after the results are known’. This questionable research practice has received increased attention in recent years because it is thought to have contributed to low …

The Preregistration Prescriptiveness Trade-Off and Unknown Unknowns in Science: Comments on Van Drimmelen (2023)

I discuss Van Drimmelen’s (2023) Metascience2023 presentation on researchers’ decision making during the research process. In particular, I consider his evidence that researchers’ discretion over research decisions is unavoidable when they follow …
JUST-OS