Publication bias (File Drawer Problem)
Definition: The failure to publish results based on the “direction or strength of the study findings” (Dickersin & Min, 1993, p. 135). The bias arises when the evaluation of a study’s publishability disproportionately hinges on the outcome of the study, often with the inclination that novel and significant results are worth publishing more than replications and null results. This bias typically materializes through a disproportionate number of significant findings and inflated effect sizes. This process leads to the published scientific literature not being representative of the full extent of all research, and specifically underrepresents null finding. Such findings, in turn, land in the so called “file drawer”, where they are never published and have no findable documentation.
Related terms: Dissemination bias, P-curve, P-hacking, Selective reporting, Statistical significance, Trim and fill **Alternative definition:** In the context of meta-analysis, publication bias “...occurs whenever the research that appears in the published literature is systematically unrepresentative of the population of completed studies. Simply put, when the research that is readily available differs in its results from the results of all the research that has been done in an area, readers and reviewers of that research are in danger of drawing the wrong conclusion about what that body of research shows.” (Rothstein et al., 2005, p. 1\) **Related terms to alternative definition:** meta-analysis
References:
- Dickersin, K., & Min, Y. (1993). Publication Bias: The Problem That Won’t Go Away. Annals of the New York Academy of Sciences, 703(1), 135–148. https://doi.org/10.1111/j.1749-6632.1993.tb26343.x
- Devito, N., & Goldacre, B. (2019). Publication Bias. Catalogue of Bias. https://catalogofbias.org/biases/publication-bias/
- Duval, S., & Tweedie, R. (2000). A nonparametric “trim and fill” method of accounting for publication bias in meta-analysis. Journal of the American Statistical Association, 95, 89–98. https://doi.org/10.2307/2669529
- Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56, 455–463. https://doi.org/10.1111/j.0006-341x.2000.00455.x
- Franco, A., Malhotra, N., & Simonovits, G. (2014). Publication bias in the social sciences: Unlocking the file drawer. Science, 345(6203), 1502–1505. https://doi.org/10.1126/science.1255484
- Lindsay, D. S. (2020). Seven steps toward transparency and replicability in psychological science. Canadian Psychology/Psychologie Canadienne, 61(4), 310–317. https://doi.org/10.1037/cap0000222
- Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2005). Publication bias in meta-analysis. In Publication bias in meta-analysis: Prevention, assessment and adjustments (pp. 1–7). John Wiley & Sons, Ltd. https://doi.org/10.1002/0470870168.ch1
Drafted and Reviewed by: Mahmoud Elsherif, Jamie P. Cockcroft, Gilad Feldman, Adrien Fillon, Helena Hartmann, Tamara Kalandadze, William Ngiam, Martin Vasilev, Olmo van den Akker, Flávio Azevedo