Replication Package for the paper: Verification of Distributed Systems via Sequential Emulation

This package allows to replicate the experiments described in “Verification of Distributed Systems via Sequential Emulation”.

Instructions for full replication are also included.

Repository contents

• main.sh: Quickstart script to run all experiments

• permissions.sh: script to give adequate permissions to executables

• replication.tar.gz: gzipped archive containing:

  • emergence, invariant, stock: C programs used in the experimental evaluation. Each directory also contains a _logs folder with the log files referred in the paper.

  • labs: LAbS specifications used for emergence and invariant experiments.

  • LICENSE: conditions to use the repository contents

  • tools: Software used to generate C programs and run experiments

  • README.mdown: the README file of the original replication repository (https://gitlab.inria.fr/ldistefa/tosem-artifacts)

  • README-ZENODO.mdown: this document

  • replication.py: Python script to replicate the experiments

  • replication.sh: fallback shell script to replicate the experiments

Requirements

All experiments should run successfully on most x64 Linux machines. Python 3.7 or above is needed to run the replication.py script. Alternatively, one may run the replication.sh shell script.

Some caveats about specific tools:

• Symbiotic requires a python executable in the machine’s PATH (usually /usr/bin/python). This may be a symbolic link pointing to a Python 3 executable (usually /usr/bin/python3).

• SeaHorn requires a Python 2 executable (usually /usr/bin/python2).

• To generate instrumented programs (see below), an external C preprocessor is required. For instance, we use gcc -E. We do provide ready-to-use instumented programs.

Quickstart

Download the whole repository to some directory dir, then:

cd /path/to/dir
# Gives execution permission to main.sh, tools, and their dependencies
chmod +x permissions.sh 
# Unpacks the archive, makes log directories, and runs all experiments
./main.sh

The script will create a dir/logs directory with subdirectories emergence, invariant, and stock, where it will store the output of each verification task.

Replication details

Generation of C programs from LAbS specifications

The following commands will use sliver to create C emulation programs from each LAbS system considered in the paper:

# Invariant programs
tools/sliver-v1.5_linux_x64/sliver.py labs/formation.invariant.labs size=10 range=2 n=3 --fair --no-bv --show > invariant/formation/formation.c
tools/sliver-v1.5_linux_x64/sliver.py labs/approx.labs yes=1 no=2 --no-bv --show > invariant/approx-a/approx-a.c
tools/sliver-v1.5_linux_x64/sliver.py labs/approx.labs yes=2 no=3 --no-bv --show > invariant/approx-a/approx-b.c.orig
tools/sliver-v1.5_linux_x64/sliver.py labs/maj.labs yes=1 no=2 --no-bv --show > invariant/maj/maj.c.orig
# Emergence programs
tools/sliver-v1.5_linux_x64/sliver.py labs/boids.labs birds=3 size=5 delta=5 --fair --no-bv --show > emergence/boids/boids.c
tools/sliver-v1.5_linux_x64/sliver.py labs/flock.labs birds=3 size=5 delta=5 --fair --no-bv --show > emergence/flock/flock.c
tools/sliver-v1.5_linux_x64/sliver.py labs/formation.emergence.labs size=10 range=2 n=3 --fair --no-bv --show > emergence/formation/formation.c

Each folder under invariant and emergence contains a <program>.c file which is the output of one the above commands. Notice that, for approx-b and maj, we have manually performed some additional transformations. The unmodified sliver output is named maj.c.orig and the final program is named maj.c.

Instrumentation of C programs

SLiVER was originally tailored towards BMC verification, so it generates programs that can be directly verified by CBMC. Other tools require an instrumentation steps. Typically this amounts to replace verification intrinsics with those supported by the tools, add specific headers or function declarations, etc. We have automated this steps by means of a helper Python program called absentee, which can be found in the tools directory and at (https://github.com/lou1306/absentee)

Typically, to instrument the SLiVER program program.c for verification by a specific tool, one can use the following command line:

cd tools/absentee
gcc -E /path/to/<program>.c | python3 -m absentee --conf <tool>.conf - > /path/to/<program>.<tool>.c

Preprocessing the original program with gcc (or another compiler) is needed because absentee does not support preprocessor directives.

Each folder under invariant and emergence already contains a file <program>.<tool>.c, which is the result of this step for each tool in our experimental evaluation.

Verification

Each tool needs a specific set of arguments, environment variables, etc. to correctly perform its verification task. To investigate the actual commands we used in our evaluation, please check the replication.sh script or the replication.py program.

About the stock case studies

We wrote the stock-safe and stock-unsafe case studies directly as triple structure, and then encoded it as C programs using the same procedure described in the paper and implemented by sliver.

We provide these C programs, as well as their instumented version for 2ls, in the stock folder.

Usage instructions for replication.py

In case one wants more granularity, replication.py may be run manually. ./replication.py --help shows usage instructions. In brief:

• ./replication.py runs all the experiments.

• ./replication.py tool1 tool2 ... only runs the experiments with the given tools.

• ./replication.py --list lists the tool names.

• ./replication --timeout <n> overrides the default timeout to n seconds (n must be a positive integer)

By adding --dry-run, the script only shows the experiments’ command lines, without actually executing them. Keep in mind that most tools also require custom environment variables or won’t run properly when invoked outside of their own directories, so these command lines are only for reference.

Support

For further information, you may look at the original Git repository for the artifacts: (https://gitlab.inria.fr/ldistefa/tosem-artifacts)

You can also contact the paper’s corresponding author: Luca Di Stefano luca.di-stefano@inria.fr

Acknowledgments

Work partially funded by MIUR project PRIN 2017FTXR7S IT MATTERS (Methods and Tools for Trustworthy Smart Systems).

Disclaimer

The authors of this repository do not endorse using its contents for ANY purpose besides replication of experiments presented in the paper “Verification of Distributed Systems via Sequential Emulation”. Please enquire with the paper’s corresponding author (Luca Di Stefano luca.di-stefano@inria.fr) about software packages intended for generic usage.