About this resource

    Type of resources
    Primary Source, Reading, Paper
    Education level(s)
    College / Upper Division (Undergraduates)
    Primary user(s)
    Student
    Subject area(s)
    Applied Science, Math & Statistics, Social Science
    Language(s)
    English
    Cluster
    Conceptual and Statistical Knowledge
    Last updated
    6 June 2020
    Primary SourceReadingPaper

    Better P-curves: Making P-curve Analysis More Robust to Errors, Fraud, and Ambitious P-hacking, a Reply to Ulrich and Miller (2015)

    By Uri Simonsohn, Joseph P Simmons, Leif D Nelson

    Open this resource

    Level College / Upper Division (Undergraduates) · For Student · Language English

    Abstract

    When studies examine true effects, they generate right-skewed p-curves, distributions of statistically significant results with more low (.01 s) than high (.04 s) p values. What else can cause a right-skewed p-curve? First, we consider the possibility that researchers report only the smallest significant p value (as conjectured by Ulrich & Miller, 2015), concluding that it is a very uncommon problem. We then consider more common problems, including (a) p-curvers selecting the wrong p values, (b) fake data, (c) honest errors, and (d) ambitiously p-hacked (beyond p < .05) results. We evaluate the impact of these common problems on the validity of p-curve analysis, and provide practical solutions that substantially increase its robustness.