UPDATE: Zenodo migration postponed to Oct 13 from 06:00-08:00 UTC. Read the announcement.

Report Open Access

Getting Research Software to Work: A Case Study on Artifact Evaluation for OOPSLA 2019

Dahlgren, Erin

Due to new peer-review programs, researchers in certain fields can now receive badges on their papers that reward them for writing functional and reusable research code. These badges in turn make their research more attractive for
others to cite and build upon. Unfortunately, some submissions to these new programs do not pass the lowest bar, and many submissions are difficult for reviewers to simply setup and test. To understand how to improve submissions and how to help researchers gain badges, we studied the artifact evaluation process of OOPSLA 2019, an ACM conference on the analysis and design of computer programs. Based on reviewer experiences, we highlight best practices and we discuss whether guidelines, tools, or larger cooperative efforts are required to achieve them. To conclude, we present ongoing and future work that helps researchers share and use research code.

Files (130.5 kB)
Name Size
130.5 kB Download
All versions This version
Views 241241
Downloads 144144
Data volume 18.8 MB18.8 MB
Unique views 216216
Unique downloads 130130


Cite as