Published May 14, 2021 | Version v1
Report Open

How reproducible should research software be?

  • 1. UK Centre for Ecology & Hydrology
  • 2. University of Oxford
  • 3. University of Hull
  • 4. University of Manchester
  • 5. University of Hertfordshire


In this article, we attempt to answer the question of how reproducible research software should be by defining four levels of reproducibility, suggesting criteria to help you decide which level your research software should be at, and recommending practices to reach these levels of reproducibility. The article is the result of a discussion session at the Software Sustainability Institute Fellows Online Selection Day 2021.



Files (209.0 kB)

Name Size Download all
209.0 kB Preview Download

Additional details


  • Wilson et al (2017). Good enough practices in scientific computing. doi: 10.1371/journal.pcbi.1005510
  • Lee et al (2021). Barely sufficient practices in scientific computing. doi: 10.1016/j.patter.2021.100206
  • McArthur (2019). Repeatability, Reproducibility, and Replicability: Tackling the 3R challenge in biointerface science and engineering. doi: 10.1116/1.5093621