Do Journal Code-sharing Policies Increase Code Availability?

Ensuring reproducibility is one of the main goals of open science. To achieve reproducibility of scientific results, data and analytical computer code (if used) should ideally be openly available. Thus far, efforts have mostly focused on making research data open and FAIR (Findable, Accessible, Interoperable, and Reusable), whereas code-sharing has only recently started to gain attention.

A recent review of the state of code availability in ecological journals with code-sharing policies showed that, despite the policies, code-sharing is alarmingly low, suggesting that those policies are not adhered to by most authors. This project builds on that recent review to explicitly test whether journal code-sharing policies increase code availability. For that, we compare code availability between 14 journals with (n = 346 articles) and 13 journals without code-sharing policies (n = 350 articles) for the period between 2015 and 2019. Our study aims to provide essential information to help improve code-sharing and journal code-sharing policies in ecology and other fields, with the ultimate goal of making science more reproducible in the short- and long-term.

Our findings thus far provide evidence that the implementation of a journal code-sharing policy does not result in a substantial proportion of code-sharing. However, it does appear to contribute to an overall increase in the availability of code, which is a positive outcome. When considering eligible articles published, the percentage of papers that shared their code was significantly higher in journals with a code-sharing policy (27%) compared to journals without such a policy (3%). Another noteworthy observation is the progressive increase in code-sharing within journals lacking a code-sharing policy over time. Specifically, only 1% of papers published between 2015-2016 shared their code, whereas this figure rose to 4% between 2018-2019. In sum, our results suggest that although code-sharing policies lead to an increase in code availability, the scientific community still needs to ensure that those policies are truly implemented. For that, we suggest that journals must provide better support for researchers and have a system for ensuring that policies are adhered to.