This chapter considers various factors that have been responsible for the comparatively slow development of psychology into a cumulative empirical science. Special attention is devoted to correctable methodological mistakes, the over-reliance upon …
This chapter considers problems with null hypothesis significance testing (NHST). The literature in this area is quite large. D. Anderson, Burnham, and W. Thompson (2000) recently found more than 300 articles in different disciplines about the …
The current crisis in scientific psychology about whether our findings are irreproducible was presaged years ago by Tversky and Kahneman (1971), who noted that even sophisticated researchers believe in the fallacious Law of Small Numbers—erroneous …
In a direct replication, the typical goal is to reproduce a prior experimental result with a new but comparable sample of participants in a high-powered replication study. Often in psychology, the research to be replicated involves a sample of …
When studies examine true effects, they generate right-skewed p-curves, distributions of statistically significant results with more low (.01 s) than high (.04 s) p values. What else can cause a right-skewed p-curve? First, we consider the …
Vul, Harris, Winkielman, and Pashler (2009), (this issue) argue that correlations in many cognitive neuroscience studies are grossly inflated due to a widespread tendency to use nonindependent analyses. In this article, I argue that Vul et al.'s …
Publication bias leads consumers of research to observe a selected sample of statistical estimates calculated by producers of research. We calculate critical values for statistical significance that could help to adjust after the fact for the …
The concept of validity has suffered because the term has been used to refer to 2 incompatible concerns: the degree of support for specified interpretations of test scores (i.e., intended score meaning) and the degree of support for specified …
Statistical rituals largely eliminate statistical thinking in the social sciences. Rituals are indispensable for identification with social groups, but they should be the subject rather than the procedure of science. What I call the “null ritual” …
Criticisms of null-hypothesis significance tests (NHSTs) are reviewed. Used as formal, two-valued decision procedures, they often generate misleading conclusions. However, critics who argue that NHSTs are totally meaningless because the null …