Planned intervention: On Wednesday April 3rd 05:30 UTC Zenodo will be unavailable for up to 2-10 minutes to perform a storage cluster upgrade.

There is a newer version of the record available.

Published September 4, 2021 | Version CCEGAR-artifact-ICSE22-submission.full
Software Open

Reproduction Package (Full Version) for ICSE 2022 Article `Decomposing Software Verification into Off-the-Shelf Components: An Application to CEGAR'

  • 1. LMU Munich
  • 2. University of Oldenburg

Description

[For convenient download we also provide a reduced version of this archive: https://doi.org/10.5281/zenodo.5301637.]

Decomposing Software Verification into Off-the-Shelf Components: An Application to CEGAR

Abstract: Tools for software verification are typically cohesive units of software with tightly coupled components. This makes it difficult to re-use components, and the potential for workload distribution is limited. Innovations in software verification might find their way into practice faster if provided in smaller, more specialized components.

In this paper, we propose to strictly decompose software verification: the verification task is split into independent subtasks, implemented by only loosely coupled components communicating via clearly defined interfaces. We apply this decomposition concept to one of the most frequently employed techniques in software verification: counterexample-guided abstraction refinement (CEGAR). CEGAR is a technique to iteratively compute an abstract model of the system. We develop a decomposition of CEGAR into independent components with clearly defined interfaces that are based on existing, standardized exchange formats.

We implement this component-based CEGAR (C-CEGAR) to investigate the feasibility of decomposition for verification. The decomposition concerns the three core tasks of CEGAR: abstract-model exploration, feasibility check, and precision refinement. We experimentally show that — despite the necessity of exchanging complex data via interfaces — the efficiency thereby only reduces by a small constant factor while the precision in solving verification tasks even increases. We furthermore illustrate the advantages of C-CEGAR by experimenting with different implementations of components, thereby further increasing the overall effectiveness and testing that substitution of components works well.

 

Contents

This document supports the following claims of our submission:

  1. The boolean expression 1 <= y + 2 ((y * -1 + 1) / 2)* is equivalent to (y mod 2 = 1)
  2. Ultimate Automizer is the only reasonable choice for invariant generator, next to CPAchecker.
  3. All data and experimental results mentioned.

In addition, all used tools are provided as archives in directory used-tools/, and the benchmark tasks used are provided in sv-benchmarks.zip.

The changes done to the wrapper script of Ultimate Automizer (as described in Sect. 5.2) are listed in ultimate-automizer-wrapper-changes.diff.

Equivalence of Boolean Expressions

See reasoning-predicates.pdf for a reasoning about the equivalence of the boolean expressions 1 <= y + 2 ((y * -1 + 1) / 2)* and (y mod 2 = 1) (according to C language semantics).

Choice of Invariant Generators

Page overview-verifiers.html gives an overview of all considered verifiers and our selection criteria.

Experiment Data

Raw Data

All experimental raw data can be found in directory raw-data/. Page overview-data.table.html gives an overview on all benchmark results (also available as CSV).

Reproducing Figures

To reproduce all figures of the paper, use the prepared files available in directory figures/.

This requires Python 3.8 and the following python modules:

  • numpy
  • matplotlib
  • scipy

You can install these modules with pip install numpy matplotlib scipy.

Python script create_figures.py produces the plots and figures of our submission from the CSV files iterations.table.csv (comparison between Pred and C-Pred) and rq3.csv (comparison between the used C-PredWit configurations).

To reproduce all files, run on the command line:

> cd figures
> python3 create_figures.py

As a result, multiple files should appear in directory figures/:

  • boxplot.pdf: Figure 10b
  • boxplot-with-outliers.pdf: the boxplot of Figure 10b, but outliers are included.
  • rq3.pdf: Figure 13
  • runtime-to-it-median.pdf: Figure 11
  • runtime-to-it-median-with-outliers.pdf: Figure 11, but outliers are included
  • scatter.pred-vs-mpred.pdf: Figure 10a

Files

CCEGAR-artifact-ICSE22-submission.full.zip

Files (56.1 GB)

Name Size Download all
md5:f270778147ef17041eb842575c212a38
56.1 GB Preview Download