Identification of Performance Changes at Code Level (Measurement Configuration Dataset)

Measurement Configuration Dataset

This is the anonymous reviewing version; the source code repository will be added after the review.

This dataset provides reproduction data for performance measurement configuration at source code level in Java. The measurement data can be obtained using the precision-experiments repository https://anonymous.4open.science/r/precision-experiments-C613/ (Examining Different Repetition Counts) yourself. These data conatained here are the data we obtained from execution on i7-4770 CPU @ 3.40GHz.

The analysis was tested on Ubuntu 20.04 and gnuplot 5.2.8. It will not work with older gnuplot versions.

To execute the analysis, extract the data by

tar -xvf basic-parameter-comparison.tar
tar -xvf parallel-sequential-comparison.tar

and afterwards build the precision-experiments repo and execute the analysis by

cd precision-experiments/precision-analysis/
../gradlew fatJar
cd scripts/configuration-analysis/
./executeCompleteAnalysis.sh ../../../../basic-parameter-comparison ../../../../parallel-sequential-comparison

Afterwards, the following files will be present:

• precision-experiments/precision-analysis/scripts/configuration-analysis/repetitionHeatmaps/heatmap_all_en.pdf (Heatmaps for different repetition counts)
• precision-experiments/precision-analysis/scripts/configuration-analysis/repetitionHeatmaps/heatmap_outlierRemoval_en.pdf (Heatmap with and without outlier removal for 1000 repetitions)
• precision-experiments/precision-analysis/scripts/configuration-analysis/histogram_outliers_en.pdf (Histogram of the outliers)
• precision-experiments/precision-analysis/scripts/configuration-analysis/heatmap_parallel_en.pdf (Heatmap with sequential and parallel execution)