I present an ontology of criteria for evaluating theory to answer the titular question from the perspective of a scientist practitioner. Set inside a formal account of our adjudication over theories, a metatheoretical calculus, this ontology …
This chapter considers various factors that have been responsible for the comparatively slow development of psychology into a cumulative empirical science. Special attention is devoted to correctable methodological mistakes, the over-reliance upon …
This chapter considers problems with null hypothesis significance testing (NHST). The literature in this area is quite large. D. Anderson, Burnham, and W. Thompson (2000) recently found more than 300 articles in different disciplines about the …
In this course, we will explore the so‐called “reproducibility crisis” that has struck fields from psychology and economics to ecology and cancer biology. You will learn statistical principles at the heart of the reproducibility crisis, how disregard …
When designing a study, the planned sample size is often based on power analyses. One way to choose an effect size for power analyses is by relying on pilot data. A-priori power analyses are only accurate when the effect size estimate is accurate. In …
This paper explains how to select the correct statistical test for a research project, clinical trial, or other investigation. The first step is to decide in what scale of measurement your data are as this will affect your decision—nominal, ordinal, …
Low power in experimental psychology is an oft-discussed problem. We show in the context of the Replicability Project: Psychology (Open Science Collaboration, 2015) that sample sizes are so small in psychology that often one cannot detect even large …
There is increasing concern that most current published research findings are false. The probability that a research claim is true may depend on study power and bias, the number of other studies on the same question, and, importantly, the ratio of …