Publication bias (File Drawer Problem)
Definition: The failure to publish results based on the “direction or strength of the study findings” (Dickersin & Min, 1993, p. 135). The bias arises when the evaluation of a study’s publishability disproportionately hinges on the outcome of the study, often with the inclination that novel and significant results are worth publishing more than replications and null results. This bias typically materializes through a disproportionate number of significant findings and inflated effect sizes. This process leads to the published scientific literature not being representative of the full extent of all research, and specifically underrepresents null finding. Such findings, in turn, land in the so called “file drawer”, where they are never published and have no findable documentation.
Alternative definition: In the context of meta-analysis, publication bias “…occurs whenever the research that appears in the published literature is systematically unrepresentative of the population of completed studies. Simply put, when the research that is readily available differs in its results from the results of all the research that has been done in an area, readers and reviewers of that research are in danger of drawing the wrong conclusion about what that body of research shows.” (Rothstein et al., 2005, p. 1)
Related terms: Dissemination bias, P-curve, P-hacking, Selective reporting, Statistical significance, Trim and fill
Related term to alternative definition: meta-analysis
References: Dickersin and Min (1993), Devito and Goldacre (2019), Duval and Tweedie (2000a, 2000b), Franco et al. (2014), Lindsay (2020), & Rothstein et al. (2005)
Drafted and Reviewed by: Mahmoud Elsherif, Jamie P. Cockcroft, Gilad Feldman, Adrien Fillon, Helena Hartmann, Tamara Kalandadze, William Ngiam, Martin Vasilev, Olmo van den Akker, Flávio Azevedo