Published November 8, 2016
| Version v1
Dataset
Open
From Reproducibility Problems to Improvements: A journey
Description
Reproducibility and repeatability are key properties of benchmarks. However, achieving reproducibility can
be difficult. We faced this while applying the microbenchmark MooBench to the resource monitoring framework SPASS-meter. In this paper, we discuss some interesting problems that occurred while trying to reproduce previous benchmarking results. In the process of reproduction, we extended MooBench and made improvements to the performance of SPASS-meter. We conclude with lessons learned for reproducing (micro-)benchmarks.
Notes
This folder contains the archived experimental data for the corresponding publication on the Symposium on Software Performance (SSP'16, http://www.performance-symposium.org/2016/program/). Please refer to benchmark.pdf or benchmark.docx for more information on the individual files and their usage.
Files
benchmark.pdf
Files
(27.8 GB)
| Name | Size | Download all |
|---|---|---|
|
md5:c991e28b55135ef0aaa2e65aa8e65e70
|
7.6 kB | Download |
|
md5:52d278840bfbab9f55bc36c71ff21cca
|
18.3 kB | Download |
|
md5:4131ded4865e8d53c67e186ce2ef71e0
|
35.5 kB | Preview Download |
|
md5:de6d5b4b982af201a60c80413df97473
|
190.2 MB | Preview Download |
|
md5:d4993cc18b3b7ac5abc80c9b98bc60f0
|
190.2 MB | Preview Download |
|
md5:140c2bd9afd511013d6577d0eb86b22f
|
972.3 MB | Preview Download |
|
md5:22b7601b1482d2bac2232fb38e901ea7
|
2.5 GB | Preview Download |
|
md5:b01da8d969e7b31aec5732d041f80abe
|
171.9 MB | Preview Download |
|
md5:84643f4bf89ebff9246775c0147e538e
|
73.3 MB | Preview Download |
|
md5:6c6ac383bd271f477ee2ac2365175541
|
511.2 MB | Preview Download |
|
md5:d8ec6027941b3ecbe0076d24c12cf7c8
|
4.1 GB | Download |
|
md5:7e00da52b57c6aba3b8709368cbcd357
|
4.1 GB | Download |
|
md5:50ab15c51e277938a1028ec8b698fa12
|
4.1 GB | Download |
|
md5:dc34d025220b8f273e513c84e3e764d1
|
4.1 GB | Download |
|
md5:dabd619b13b9afde07fad42a9e006af0
|
4.1 GB | Download |
|
md5:db025bc72d135cde29ce1d17b3135f21
|
2.7 GB | Download |
|
md5:f2a4e15de672f2940f5416535ca10db5
|
5.9 kB | Preview Download |
|
md5:0f2a531ed4a66738b69faabf6602a24c
|
37.7 MB | Preview Download |
Additional details
Funding
References
- H. Eichelberger, A. Sass, K. Schmid, From Reproducibility Problems to Improvements: A journey, Symposium on Software Performance (SSP '16), 2016