Throughout their education and when reading the scientific literature, students may get the impression that there is a unique and correct analysis strategy for every data analysis task and that this analysis strategy will always yield a significant …
The .05 boundary within Null Hypothesis Statistical Testing (NHST) “has made a lot of people very angry and been widely regarded as a bad move” (to quote Douglas Adams). Here, we move past meta-scientific arguments and ask an empirical question: What …
Campbell’s Law explains the replication crisis. In brief, useful tools such as hypotheses, p-values, and multi-study designs came to be viewed as indicators of strong science, and thus goals in and of themselves. Consequently, their use became …
Survey experiments often yield intention-to-treat effects that are either statistically and/or practically “non-significant.” There has been a commendable shift toward publishing such results, either to avoid the “file drawer problem” and/or to …
We assess statistical power and excess statistical significance among 31 leading economics general interest and field journals using 22,281 parameter estimates from 368 distinct areas of economics research. Median statistical power in leading …
This book starts from the premise that there is a lot we can all do to increase the benefits of research.
Let’s consider the main limitations of research that is not carried out and shared in an open, transparent, and reproducible way:
If papers …
Last Contact is a game is a game that is intended to inform people about the open science movement. In the game players encounter common problems of current scientific practice in form of a parody science fiction scenario. The problems that players …
Ecological momentary assessment (EMA) represents a promising approach to study cognitive aging. In contrast to laboratory-based studies, EMA involves the repeated sampling of experiences in daily life contexts, enabling investigators to gain access …
Validity evidence based on test content is critical to meaningful interpretation of test scores. Within high-stakes testing and accountability frameworks, content-related validity evidence is typically gathered via alignment studies, with panels of …
Qualitative training rarely acknowledges the role of emotions in both data collection and analysis. While bracketing emotions is an important part of reflexivity, emotions are both a source of data and a source of ‘work’ (Hochschild, Citation1983). …