Publication bias (File Drawer Problem)
Definition: The failure to publish results based on the “direction or strength of the study findings” (Dickersin & Min, 1993, p. 135). The bias arises when the evaluation of a study’s publishability disproportionately hinges on the outcome of the study, often with the inclination that novel and significant results are worth publishing more than replications and null results. This bias typically materializes through a disproportionate number of significant findings and inflated effect sizes. This process leads to the published scientific literature not being representative of the full extent of all research, and specifically underrepresents null finding. Such findings, in turn, land in the so called “file drawer”, where they are never published and have no findable documentation.
Related terms: Dissemination bias, P-curve, P-hacking, Selective reporting, Statistical significance, Trim and fill, Meta-Analysis
References:
- Dickersin, K., & Min, Y. (1993). Publication Bias: The Problem That Won’t Go Away. Annals of the New York Academy of Sciences, 703(1), 135–148. https://doi.org/10.1111/j.1749-6632.1993.tb26343.x
- Devito, N., & Goldacre, B. (2019). Publication Bias. Catalogue of Bias. https://catalogofbias.org/biases/publication-bias/
- Duval, S., & Tweedie, R. (2000). A nonparametric “trim and fill” method of accounting for publication bias in meta-analysis. Journal of the American Statistical Association, 95, 89–98. https://doi.org/10.2307/2669529
- Duval, S., & Tweedie, R. (2000). Trim and fill: A simple funnel-plot–based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56, 455–463. https://doi.org/10.1111/j.0006-341x.2000.00455.x
- Franco, A., Malhotra, N., & Simonovits, G. (2014). Publication bias in the social sciences: Unlocking the file drawer. Science, 345(6203), 1502–1505. https://doi.org/10.1126/science.1255484
- Lindsay, D. S. (2020). Seven steps toward transparency and replicability in psychological science. Canadian Psychology/Psychologie Canadienne, 61(4), 310–317. https://doi.org/10.1037/cap0000222
- Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2005). Publication bias in meta-analysis. In Publication bias in meta-analysis: Prevention, assessment and adjustments (pp. 1–7). John Wiley & Sons, Ltd. https://doi.org/10.1002/0470870168.ch1
Originally drafted by: Mahmoud Elsherif
Reviewed by: Jamie P. Cockcroft, Gilad Feldman, Adrien Fillon, Helena Hartmann, Tamara Kalandadze, William Ngiam, Martin Vasilev, Olmo van den Akker, Flávio Azevedo