Journal article Open Access

What p -hacking really looks like: A comment on Masicampo and LaLande (2012)

Lakens, Daniël


JSON Export

{
  "files": [
    {
      "links": {
        "self": "https://zenodo.org/api/files/2a595218-b6f5-4983-b766-3b0691b63790/document_VbiA8AH.pdf"
      }, 
      "checksum": "md5:40aeefe4dde04fb47d04d3e8c21552f3", 
      "bucket": "2a595218-b6f5-4983-b766-3b0691b63790", 
      "key": "document_VbiA8AH.pdf", 
      "type": "pdf", 
      "size": 185894
    }
  ], 
  "owners": [
    13380
  ], 
  "doi": "10.1080/17470218.2014.982664", 
  "stats": {
    "version_unique_downloads": 201.0, 
    "unique_views": 384.0, 
    "views": 398.0, 
    "version_views": 399.0, 
    "unique_downloads": 201.0, 
    "version_unique_views": 385.0, 
    "volume": 38665952.0, 
    "version_downloads": 208.0, 
    "downloads": 208.0, 
    "version_volume": 38665952.0
  }, 
  "links": {
    "doi": "https://doi.org/10.1080/17470218.2014.982664", 
    "latest_html": "https://zenodo.org/record/235811", 
    "bucket": "https://zenodo.org/api/files/2a595218-b6f5-4983-b766-3b0691b63790", 
    "badge": "https://zenodo.org/badge/doi/10.1080/17470218.2014.982664.svg", 
    "html": "https://zenodo.org/record/235811", 
    "latest": "https://zenodo.org/api/records/235811"
  }, 
  "created": "2017-01-09T15:25:05.166863+00:00", 
  "updated": "2020-01-20T17:29:11.200703+00:00", 
  "conceptrecid": "721889", 
  "revision": 6, 
  "id": 235811, 
  "metadata": {
    "access_right_category": "success", 
    "doi": "10.1080/17470218.2014.982664", 
    "description": "Masicampo and Lalande (2012; M&L) assessed the distribution of 3627 exactly calculated p-values between 0.01 and 0.10 from 12 issues of three journals. The authors concluded that \"The number of p-values in the psychology literature that barely meet the criterion for statistical significance (i.e., that fall just below .05) is unusually large\". \"Specifically, the number of p-values between .045 and .050 was higher than that predicted based on the overall distribution of p.\"\nThere are four factors that determine the distribution of p-values, namely the number of studies examining true effect and false effects, the power of the studies that examine true effects, the frequency of Type 1 error rates (and how they were inflated), and publication bias. Due to publication bias, we should expect a substantial drop in the frequency with which p-values above .05 appear in the literature. True effects yield a right-skewed p-curve (the higher the power, the steeper the curve, e.g., Sellke, Bayarri, & Berger, 2001). When the null-hypothesis is true the p-curve is uniformly distributed, but when the Type 1 error rate is inflated due to flexibility in the data-analysis, the p-curve could become left-skewed below pvalues of .05.\nM&L (and others, e.g., Leggett, Thomas, Loetscher, & Nicholls, 2013) model pvalues based on a single exponential curve estimation procedure that provides the best fit of p-values between .01 and .10 (see Figure 3, right pane). This is not a valid approach because p-values above and below p=.05 do not lie on a continuous curve due to publication bias. It is therefore not surprising, nor indicative of a prevalence of p-values just below .05, that their single curve doesn't fit the data very well, nor that Chi-squared tests show the residuals (especially those just below .05) are not randomly distributed.", 
    "license": {
      "id": "other-open"
    }, 
    "title": "What p -hacking really looks like: A comment on Masicampo and LaLande (2012)", 
    "relations": {
      "version": [
        {
          "count": 1, 
          "index": 0, 
          "parent": {
            "pid_type": "recid", 
            "pid_value": "721889"
          }, 
          "is_last": true, 
          "last_child": {
            "pid_type": "recid", 
            "pid_value": "235811"
          }
        }
      ]
    }, 
    "publication_date": "2014-12-06", 
    "creators": [
      {
        "orcid": "0000-0002-0247-239X", 
        "name": "Lakens, Dani\u00ebl"
      }
    ], 
    "access_right": "open", 
    "resource_type": {
      "subtype": "article", 
      "type": "publication", 
      "title": "Journal article"
    }
  }
}
398
208
views
downloads
Views 398
Downloads 208
Data volume 38.7 MB
Unique views 384
Unique downloads 201

Share

Cite as