Effects and Environments of Reproducibility Projects

Problems of replicability of probabilistic findings have been discussed since Ioannidis. In psychology, the debate was triggered in particular by the controversy surrounding negative replication attempts of a study with highly anomalous results in social psychology. A broader disciplinary and public debate accelerated through several smaller reproducibility projects, notably the Reproducibility Project: Psychology (RP:P) organised by the Open Science Collaboration (2015), which achieved only moderate replication success. Both intra-disciplinary and inter-disciplinary communities responded to this epistemic-methodological crisis by engaging in statistical methodological discourses and further monitoring. A number of questionable research practices, in particular selective reporting, the tendency towards small sample and effect sizes, but also weaknesses in theory development and contextual sensitivity  have been suggested as causes. These different findings show that failed replications are a complex and not per se unambiguous phenomenon.

This paper aims to answer these questions in an exploratory way by analysing citation dynamics and discursive patterns in citation contexts.