# Artifact for paper "CBMC-SSM: Bounded Model Checking of C Programs with Symbolic Shadow Memory"
by Bernd Fischer, Salvatore La Torre, Gennaro Parlato, and Peter Schrammel


## Overview

The goal of this artifact is to demonstrate the use of symbolic shadow memory as implemented 
in CBMC-SSM.

The artifact contains the following files:

Readme.txt               this file
License.txt              the license of CBMC covering the binaries and scripts

taint/example.c          the taint example in Figure 1 of the paper
taint/run.sh             a script for running CBMC-SSM on the taint example
taint/run.log            the expected log output of run.sh

svcomp/22/...            the SV-COMP concurrency benchmarks instrumented for data race detection
svcomp/run_full.sh       a script for running CBMC-SSM on all the benchmarks
svcomp/run_short.sh      a script for running CBMC-SSM on a selected subset of the benchmarks
svcomp/expected_results_full.csv the results table as produced by running run_full.sh
svcomp/expected_results_short.csv the results table as produced by running run_short.sh
svcomp/comparison.csv    the results table of a comparison with PorSE and ThreadSanitizer on the benchmarks

bin/cbmc-ssm             the CBMC-SSM executable for linux x86_64,
                         built from https://github.com/peterschrammel/cbmc/tree/shadow-memory-demo
bin/run_svcomp.py        a helper script
bin/run_svcomp_short.py  a helper script


## Running the artifact

### General instructions

1. Open the CBMC-SSM.ova Virtual Box Image.
2. Launch the VM
3. Login with the password cbmcssmdemo

### Check that CBMC-SSM is executing

Open a terminal and type
```
cd Documents
bin/cbmc-ssm --version
```

You should see the output
```
5.64.0 (cbmc-5.64.0-118-gc0a8a9cd0a)
```


### Running CBMC-SSM on the taint example

The paper contains in Figure 1 an example of a program instrumented for taint analysis
that can be analyzed using CBMC-SSM.
To run this example, 
open a terminal and type
```
cd Documents
cd taint
./run.sh
```

The output should match the content of the file run.log
(when redirecting stdout and stderr into the file). 
In particular, the result should be `VERIFICATION SUCCESSFUL`,
meaning that the encoded JSON - as expected - contains tainted data only in the
"password" property.

Changing line 59 in example.c to
```
make_nondet_len_string(uname, 1);
```
and running ./run.sh again would lead to result `VERIFICATION FAILED` because
then also the "username" property contains tainted data, which is unexpected.


### Running CBMC-SSM on the SV-COMP concurrency benchmarks

In section 5 of the paper, we used CBMC-SSM to find data races in the 
SV-COMP benchmarks. For that purpose, we instrumented 153 SV-COMP concurrency
benchmarks according to the instrumentation described in section 5,
using the functions of the symbolic shadow memory API provided by CBMC-SSM.
The resulting files are in directory svcomp/22.
These files have already been preprocessed to perform 3 sequentialization rounds
and 4 loop unwindings as described in the paper. 
CBMC-SSM can thus be invoked with `--unwind 1` as the required loop unwindings have
already been performed.

For your convenience we have selected a subset of the benchmarks,
which can be run in less than 5 minutes by opening a terminal and 
using the commands
```
cd Documents
cd svcomp
./run_short.sh
```

The invoked CBMC-SSM commands are printed to the terminal as the script is
running.
When the script has finished a file results_short.csv is produced
that contains the results table. An entry "P" in the second column
means that there is a data race, "N" means that there is no data race.

We have collected the expected results in the file
`expected_results_short.csv`.

Running the entire benchmark set requires 6 to 8 hours on an Intel Core i7 @ 2.6 GHz
and 8GB RAM made available to the VM.
You can replicate the contents of `expected_results_full.csv` by 
running `./run_full.sh` from the svcomp directory,
which produces a `results.csv` file.
Note that these results do not exactly match the numbers in the paper,
which have been obtained on a more powerful machine.