Comparing Transparent Static Analyzers with Open Verification Dashboard (Artifact)

This artifact accompanies the paper “Comparing Transparent Static Analyzers with Open Verification Dashboard”. It bundles the Open Verification Dashboard source code together with scripts, benchmark subsets, tool binaries, and precomputed experiment outputs needed to reproduce the main reported results.

The artifact supports three main reproduction workflows. For RQ1, it reproduces the post-alignment cross-analyzer comparison of arithmetic overflow proof obligations on the SV-COMP no-overflow benchmark subset, including the aggregated matrix reported in Table 1. For RQ2, it reproduces the joint-progress-matrix example underlying Table 3b and generates the corresponding CSV output. For RQ3, it provides the filtering, reduction, and reporting pipeline used to compute the folder-level averages reported in Table 5.

The package also includes a lightweight quickstart workflow for “kick-the-tires” evaluation, allowing to run a small end-to-end pipeline before attempting the larger experiments. In addition to the experiment data, the artifact contains the full source code of the Open Verification Dashboard (OVD), which can be rebuilt and reused to compare PO outputs from other analyzers beyond the specific Goblint/Mopsa experiments in the paper.

In short, this Zenodo deposit provides both a reproducibility package for the paper’s reported results and a reusable implementation of the dashboard software that underlies the comparisons.