Data and Code for Reproducible Research: Lessons Learned from the NLM Reproducibility Workshop

The National Library of Medicine held two workshops in 2019 in which National Institutes of Health researchers attempted to reproduce the results of published bioinformatics paper that stated they made their data and code publicly available. Not a single one of the ten teams over two workshops could fully reproduce the results of the paper. Despite the teams’ inability to reproduce the results, the workshops provided valuable insights into where researchers are falling short in sharing and documenting their data and code. This presentation, prepared for csv,conf,v5, discusses some of the issues that contributed to the irreproducibility of these papers to provide a better understanding of how researchers can ensure that their science is transparent and reproducible.