Journal article Open Access

# What p -hacking really looks like: A comment on Masicampo and LaLande (2012)

Lakens, Daniël

### DCAT Export

<?xml version='1.0' encoding='utf-8'?>
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:adms="http://www.w3.org/ns/adms#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:dctype="http://purl.org/dc/dcmitype/" xmlns:dcat="http://www.w3.org/ns/dcat#" xmlns:duv="http://www.w3.org/ns/duv#" xmlns:foaf="http://xmlns.com/foaf/0.1/" xmlns:frapo="http://purl.org/cerif/frapo/" xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#" xmlns:gsp="http://www.opengis.net/ont/geosparql#" xmlns:locn="http://www.w3.org/ns/locn#" xmlns:org="http://www.w3.org/ns/org#" xmlns:owl="http://www.w3.org/2002/07/owl#" xmlns:prov="http://www.w3.org/ns/prov#" xmlns:rdfs="http://www.w3.org/2000/01/rdf-schema#" xmlns:schema="http://schema.org/" xmlns:skos="http://www.w3.org/2004/02/skos/core#" xmlns:vcard="http://www.w3.org/2006/vcard/ns#" xmlns:wdrs="http://www.w3.org/2007/05/powder-s#">
<dct:identifier rdf:datatype="http://www.w3.org/2001/XMLSchema#anyURI">https://zenodo.org/record/235811</dct:identifier>
<foaf:page rdf:resource="https://zenodo.org/record/235811"/>
<dct:creator>
<rdf:type rdf:resource="http://xmlns.com/foaf/0.1/Agent"/>
<dct:identifier rdf:datatype="http://www.w3.org/2001/XMLSchema#string">0000-0002-0247-239X</dct:identifier>
<foaf:name>Lakens, Daniël</foaf:name>
<foaf:givenName>Daniël</foaf:givenName>
<foaf:familyName>Lakens</foaf:familyName>
</rdf:Description>
</dct:creator>
<dct:title>What p -hacking really looks like: A comment on Masicampo and LaLande (2012)</dct:title>
<dct:publisher>
<foaf:Agent>
<foaf:name>Zenodo</foaf:name>
</foaf:Agent>
</dct:publisher>
<dct:issued rdf:datatype="http://www.w3.org/2001/XMLSchema#gYear">2014</dct:issued>
<dct:issued rdf:datatype="http://www.w3.org/2001/XMLSchema#date">2014-12-06</dct:issued>
<owl:sameAs rdf:resource="https://zenodo.org/record/235811"/>
<skos:notation rdf:datatype="http://www.w3.org/2001/XMLSchema#anyURI">https://zenodo.org/record/235811</skos:notation>
<owl:sameAs rdf:resource="https://doi.org/10.1080/17470218.2014.982664"/>
<dct:description>Masicampo and Lalande (2012; M&amp;amp;L) assessed the distribution of 3627 exactly calculated p-values between 0.01 and 0.10 from 12 issues of three journals. The authors concluded that "The number of p-values in the psychology literature that barely meet the criterion for statistical significance (i.e., that fall just below .05) is unusually large". "Specifically, the number of p-values between .045 and .050 was higher than that predicted based on the overall distribution of p." There are four factors that determine the distribution of p-values, namely the number of studies examining true effect and false effects, the power of the studies that examine true effects, the frequency of Type 1 error rates (and how they were inflated), and publication bias. Due to publication bias, we should expect a substantial drop in the frequency with which p-values above .05 appear in the literature. True effects yield a right-skewed p-curve (the higher the power, the steeper the curve, e.g., Sellke, Bayarri, &amp;amp; Berger, 2001). When the null-hypothesis is true the p-curve is uniformly distributed, but when the Type 1 error rate is inflated due to flexibility in the data-analysis, the p-curve could become left-skewed below pvalues of .05. M&amp;amp;L (and others, e.g., Leggett, Thomas, Loetscher, &amp;amp; Nicholls, 2013) model pvalues based on a single exponential curve estimation procedure that provides the best fit of p-values between .01 and .10 (see Figure 3, right pane). This is not a valid approach because p-values above and below p=.05 do not lie on a continuous curve due to publication bias. It is therefore not surprising, nor indicative of a prevalence of p-values just below .05, that their single curve doesn't fit the data very well, nor that Chi-squared tests show the residuals (especially those just below .05) are not randomly distributed.</dct:description>
<dct:accessRights rdf:resource="http://publications.europa.eu/resource/authority/access-right/PUBLIC"/>
<dct:accessRights>
<rdfs:label>Open Access</rdfs:label>
</dct:RightsStatement>
</dct:accessRights>
<dcat:distribution>
<dcat:Distribution>
<dcat:accessURL rdf:resource="https://doi.org/10.1080/17470218.2014.982664"/>
<dcat:byteSize>185894</dcat:byteSize>
<dcat:mediaType>application/pdf</dcat:mediaType>
</dcat:Distribution>
</dcat:distribution>
</rdf:Description>
</rdf:RDF>

396
207
views