Artifact for the paper "Modular Reasoning about Global Variables and Their Initialization"

Our artifact contains the implementation of the technique described in the paper "Modular Reasoning about Global Variables and Their Initialization" in VerCors and Gobra, the case studies, instructions on how to reproduce the evaluation, and the formalization in Iris.

Software Dependencies

We provide our artifact as a Docker image. Thus, the instructions below require that Docker is installed.

Hardware Dependencies

Our artifact does not require any specific hardware other than that found in commodity laptops and desktops. However, we recommend using a machine with a modern CPU to verify the realistic router implementation.

Getting started

To set-up the artifact image, please follow the following steps:

- Download the docker image provided as a .tar
 
- Load the docker image using the following command

        docker image load -i artifact.tar

- Afterwards, you can run the container by executing the following command

        docker run -it oopsla25/artifact-global-state-init bash

    This opens an interactive terminal where you can run all commands shown in this document.

- Make sure you can run our implementation of VerCors by running the following command in the docker container

        /artifact/bin/vct /artifact/evaluation/vercors/byte/Byte.java

    You should get a success message like

        [INFO] Starting verification
        [INFO] Verification completed successfully.
        Verification took 5413 ms

- Make sure you can run our implementation of Gobra too, by running the following command:

        java -Xss1g -Xmx4g -jar /artifact/bin/gobra.jar -p /artifact/evaluation/gobra/byte/

    You should get the following success message:

        [main] INFO viper.gobra.Gobra - Verifying package evaluation/gobra/byte - byte
        [thread-12] INFO viper.gobra.Gobra - Gobra found 0 errors.

 

Reproducing results in VerCors

Running VerCors

To verify a file using our modified version of VerCors,

 1. Run the docker container as described above
 2. In the container, from the home directory (/artifact), run ./bin/vct <options> path/to/file.java
    For example, run ./bin/vct evaluation/vercors/byte/Byte.java to verify the Byte example from the evaluation.
    Successful verification will result in output similar to the following:
        
        [INFO] Verification completed successfully.
        Verification took 27965 ms.

Alternatively, VerCors may report one or more verification errors and (usually) where they occur. An example is 

    At examples/technical/java/Finalizer.java:72:23:
    --------------------------------------
       70  
       71          // push onto unfinalized
                                [----
       72          synchronized (lock) {
                                 ----]
       73              if (unfinalized != null) {
       74                  this.next = unfinalized;
    --------------------------------------
    There may be insufficient permission to access this field here. 

which states that there is insufficient permission to access the field “lock” in line 72 in the example.

The two command line options that are relevant for our examples are:
 - –sequential-java-init: This option enables the mode for verifying sequential Java programs described in Sec. 4.1.2 in the paper. That is, it removes the requirement to own a token to open a non-duplicable invariant, and instead imposes an additional level requirement as described in the paper.
 - –backend-file-base=name.vpr: This option instructs VerCors to emit the generated Viper code to the file name.vpr, which makes it possible to inspect how our version of VerCors encodes the proof obligations described in the paper.

Note that VerCors may additionally output warnings (typically about quantifiers that have no user-provided triggers) or additional information during verification; this output is intended for debugging and does not mean that there is a problem.

VerCors Examples

The examples from our evaluation can be found in the directory /artifact/evaluation/vercors in the artifact, where each example has its own subdirectory. The examples that are intended to be executed and verified in a sequential setting, i.e., the ones marked with a star in Table 1 in the paper, have the suffix “_sequential” in their subdirectory name.
Each subdirectory contains one or more .java files that make up the example. Note that, unlike in standard Java, VerCors allows multiple class definitions in a single .java file

VerCors Specifications

Specifications and ghost statements in VerCors are included in the source code in the form of special comments marked with the @ symbol. In particular, a precondition P is expressed by writing either 

    //@ requires P; 
or 

    /*@ requires P; */

Analogously to “requires” for preconditions, the keywords “ensures” and “loop_invariant” are used for postconditions and loop invariants. 
We have added the keywords “static_invariant” and “dup_static_invariant”, which users can add to class declarations to define the non-duplicable and duplicable invariants of the class, respectively. 

Similarly, the keyword “static_level” can be used on classes, static initializers, constructors, and methods to define their respective initialization level (in the form of a natural number). Note that, as explained in the paper, our implementation chooses default values for initialization levels to be 1 for classes and methods, and l − 1 for the initializer of a class with level l.

Additionally, we have added the following assertions:

  - “\initialized(C)” represents Init_C in the paper
  - “\token(C, p)” represents a fractional amount of p of the token of class C
  - “\onInit(C)(A)” represents the assertion A under the initialization-guarded modality for class C

We have added the following initialization-specific ghost statements:

  - “openInv C p” opens a fractional amount p of the non-duplicable invariant of class C
  - “openDupInv C” opens the duplicable invariant of class C
  - “closeInv C p” closes a fractional amount p of the non-duplicable invariant of class C

That means that using rule OpenMInv in the logic corresponds to first using the openInv ghost statement and later using the closeInv ghost statement in VerCors. Using rule OpenDupMInv in the logic corresponds to using the openDupInv ghost statement in VerCors.

Reproducing Table 1

Table 1 shows the verification times and other information about the verified Java examples.
Recall that the examples marked with a star are verified using the sequential Java mode, i.e., using the command line argument –sequential-java-init as described above.
We counted every non-empty line containing specifications or ghost code, i.e., any non-empty line starting with //@ or surrounded by /*@ … */, as a line of proof annotation (column “Ann.”). Lines containing “openInv”, “openDupInv”, “closeInv”, “static_level”, “static_invariant”, or “dup_static_invariant” annotations, as well as all other annotations that were relevant for code inside the static initializer, were counted as lines of annotations specific to initialization (column “MInv”).
To reproduce the verification times for the Java examples in Table 1, go to /artifacts/scripts, and run

    ./run-vercors-table1.sh

This script will start a JVM and run our version of VerCors five times on all examples in the table, with the –sequential-java-init flag set when appropriate, and output the verification times for each run. For example, for the Byte case study (which is verified first), you will see the following output:

    Running: /artifact/bin/ng vct.main.Main --path-z3 /usr/bin/z3 /artifact/evaluation/vercors/byte/Byte.java
    Running: /artifact/bin/ng vct.main.Main --path-z3 /usr/bin/z3 /artifact/evaluation/vercors/byte/Byte.java
    Running: /artifact/bin/ng vct.main.Main --path-z3 /usr/bin/z3 /artifact/evaluation/vercors/byte/Byte.java
    Running: /artifact/bin/ng vct.main.Main --path-z3 /usr/bin/z3 /artifact/evaluation/vercors/byte/Byte.java
    Running: /artifact/bin/ng vct.main.Main --path-z3 /usr/bin/z3 /artifact/evaluation/vercors/byte/Byte.java
    Verification times: 
    12.89, 4.48, 4.01, 3.33, 3.33,

This means that the example was verified five times, taking 12.89 seconds for the first attempt (which is due to the JVM not being warmed up for the first attempt), 4.48 seconds for the second attempt, and so on. Ignoring the first two times for the “Byte” example, which are inflated due to JVM warmup, all reported numbers should roughly match the ones shown in the paper (modulo differences due to different hardware and the use of docker).

Source code

The source code of our implementation in VerCors is publicly available at https://github.com/marcoeilers/vercors/tree/meilers_jar_assembly, and can be found at /artifact/src/vercors in the artifact.
 
VerCors is implemented as a series of encoding steps from the original Java code to a program in the much simpler Viper intermediate language, which is then verified by a Viper backend. Our encoding is mostly implemented in the file src/rewrite/vct/rewrite/EncodeStaticInitialization.scala, which represents such an encoding state. The different overloads of the “dispatch” method define how different expressions, statements and declarations are desugared into simpler ones or how initialization-related proof obligations are added.

Benchmarking results in Gobra

Running Gobra

To verify a file using our modified version of Gobra,
 1. Run the docker container as described above
 2. In the container, from the home directory (/artifact), run `java -jar bin/gobra.jar -p path/to/gobra/package`
    For example, `java -jar /bin/gobra.jar evaluation/gobra/byte/` to verify the Byte example from the evaluation.
    Successful verification will result in output similar to the following:

        [main] INFO viper.gobra.Gobra - Verifying package evaluation/gobra/byte - byte
        [thread-12] INFO viper.gobra.Gobra - Gobra found 0 errors.

    Alternatively, Gobra may report one or more verification errors and (usually) where they occur.

When Gobra cannot prove the correctness of a program (e.g., because it is incorrect), it generates verification errors. As an example, we have an incorrect version of the byte package in evaluation/gobra/byte-wrong/, which may lead to runtime panics due to array accesses out of bounds. Running  java -jar /bin/gobra.jar evaluation/gobra/byte-wrong/ leads to the following error:

    Error at: <evaluation/gobra/byte-wrong/byte.go:47:2> Assignment might fail. 
    Permission to byteCache[val] might not suffice.
    ERROR viper.gobra.Gobra - Gobra found 1 error.

This error reports that the array access byteCache[val] cannot be proven safe because no permissions to this array location were available when the access occurs. Because Gobra guarantees that permissions to invalid array indexes are never available, the permission mechanism rejects out-of-bounds accesses to arrays.

Gobra Examples

The examples from our evaluation can be found in the directory /artifact/evaluation/gobra in the artifact, where each example has its own subdirectory. Each subdirectory contains one or more .go and .gobra files that make up the example.

Gobra Specifications

Gobra accepts two kinds of files: .go and .gobra files. The former corresponds to regular Go code. In these files, similarly to Vercors, specifications and ghost statements in Gobra are included in the source code in the form of special comments marked with the @ symbol. In particular, a precondition P is expressed by writing either 

    //@ requires P
or 

    /*@ requires P @*/

.gobra files contain exclusively ghost code and thus, annotations do not occur in comments.

Analogously to “requires” for preconditions, the keywords “ensures” and “invariant” are used for postconditions and loop invariants. 
We have added the clauses “pkgInvariant” and “dup pkgInvariant”, which users can add to any .go or .gobra file, above the package clause, and which allows declaring package invariants.
 
As explained in section 4.2, Gobra automatically infers the initialization level of all packages and most methods. We introduce the keyword “mayInit”, also described in section 4.2, to mark the methods that may be run during the initialization of the package where they are declared. 

The simpler semantics of Go allows for various simplifications (section 4.2.1). Thus, contrary to Vercors, we did not add the Token and Initialized assertions to Gobra. 

We extended Gobra with the ghost statement openDupInv which (safely) assumes the duplicable invariants of the package under verification. Gobra checks that this statement only occurs after the package has been initialized, by checking the level requirement prescribed by the rule OpenDupMInv.

Finally, we introduced the notion of friend package invariants, which are declared with the keyword friendPkg right after the package clause. This feature allows us to declare certain kinds of invariants of a package A that depend on global variables of package B, when package B imports A. We note that friend package invariants have a direct translation to module invariants in a setting where the invariants of different modules are allowed to refer to each other.

Friend package invariants require that we pass the feature flag `--experimentalFriendClauses` to Gobra. They are not used in the benchmarks from section 5.1., but they are used in the verified router code described in section 5.2.

Reproducing Table 1

Table 1 shows the verification times and other information about the verified Go examples.
We counted every line starting with // @ in *.go files, as well as all lines in *.gobra files (which contain only definitions for verification purposes and no executable code) as a line of proof annotation (column Ann.). Lines specifying initializers (i.e., “init” functions) and the definitions they require, as well as lines starting with “mayInit”, “friend”, “pkgInvariant”, “dup pkgInvariant”, and “openDupPkgInvariant” lines were counted as  lines of annotations specific to initialization (column “MInv”).

To reproduce the verification times for the Go examples in Table 1, go to /artifacts/scripts, and run

    ./run-gobra-table1.sh

This script will start a JVM and run our version of Gobra five times on all examples in the table and output the verification times for each run. For example, for the Byte case study (which is verified first), you will see the following output:

    Running: /artifact/bin/ng viper.gobra.GobraRunner --mceMode=off -r --projectRoot /artifact/evaluation/gobra/byte
    Running: /artifact/bin/ng viper.gobra.GobraRunner --mceMode=off -r --projectRoot /artifact/evaluation/gobra/byte
    Running: /artifact/bin/ng viper.gobra.GobraRunner --mceMode=off -r --projectRoot /artifact/evaluation/gobra/byte
    Running: /artifact/bin/ng viper.gobra.GobraRunner --mceMode=off -r --projectRoot /artifact/evaluation/gobra/byte
    Running: /artifact/bin/ng viper.gobra.GobraRunner --mceMode=off -r --projectRoot /artifact/evaluation/gobra/byte
    Verification times: 
    6.95s 2.15s 1.93s 1.75s 1.69s 

This means that the example was verified five times, taking 6.95 seconds for the first attempt (which is due to the JVM not being warmed up for the first attempt), 2.15 seconds for the second attempt, and so on. Analogous to the VerCors times, except for the first two times for the “Byte” example, which are inflated due to JVM warmup, all reported numbers should roughly match the ones shown in the paper (modulo differences due to different hardware and the use of docker).

Reproducing the results from section 5.2

In our experiments, verifying the Go router took on average 45m46s without verifying the proof obligations related to initialization and opening module invariants, and 47m05s with them enabled. In total, enabling these proof obligations led to an increase of 79s, which is less than 3% of the original verification time, thus supporting our claim that, compared to the previous version of the code base, our changes did not slow down verification significantly.

To reproduce the verification times, we provide two scripts in /artifacts/scripts: 
  - run-gobra-router-no-init.sh
  - run-gobra-router.sh

The former is our baseline: it verifies the entire project where all initialization-related proof obligations are assumed to hold. The later script contains the project, where initialization-related proof obligations are checked.

Both scripts will start a JVM and run our version of Gobra on the packages of the project. For example, for the pkg/addr package, you will see the following output, which shows the command being executed and the total verification time:

    Running: /artifact/bin/ng viper.gobra.GobraRunner -p /artifact/evaluation/gobra/VerifiedSCION/pkg/addr --chop 1 -I /artifact/evaluation/gobra/VerifiedSCION/ /artifact/evaluation/gobra/VerifiedSCION/verification/dependencies --onlyFilesWithHeader -m github.com/scionproto/scion --mceMode=od --experimentalFriendClauses --moreJoins off
    Verification time: 6.16s
 
We observe that the verification times we report may vary significantly depending on the hardware. Crucially, for these experiments, we are concerned mostly with the relative slowdown of adding the proof obligations for initialization code.

Source code

Our extension to Gobra has been merged in the master branch of Gobra (https://github.com/viperproject/gobra/pull/810), and thus, it is publicly accessible to all users of the tool. It can be found at /artifact/src/gobra  in the artifact.

Gobra is implemented as a program that parses annotated Go programs, and generates a corresponding Viper program that verifies if and only if the original Gobra program should verify.
The new proof obligations are mostly implemented in the file src/main/scala/viper/gobra/frontend/Desugar.scala, which lowers Go programs into programs in an intermediate representation that has no notion of global state and module initializers. We recommend that interested readers check the documentation of the methods generatePkgInitProofObligations, checkPkgImportObligations, and generateMainFuncProofObligation, which implement the most important pieces of functionality.

Checking the proofs in Rocq/Iris

Building the proofs

To build the proofs and check they hold,

 1. Run the docker container as described above,
 2. In the container, run the “check-formalization-coq.sh”. This will build all the proofs in the directory from scratch, and eventually build and check the main soundness theorem. This will also print the assumptions used for the theorem.

You should see something like the following output:

    make[1]: Entering directory '/artifact/formalisation'
    CLEAN
    make[1]: Leaving directory '/artifact/formalisation'
    …
    "coq_makefile" -f _CoqProject -o Makefile.coq
    make[1]: Entering directory '/artifact/formalisation'
    COQDEP VFILES
    COQC theories/strings.v
    COQC theories/fresh.v
    COQC theories/generation.v
    COQC theories/level.v
    COQC theories/lock.v
    COQC theories/lang.v
    COQC theories/lang_notation.v
    COQC theories/assertion.v
    COQC theories/big_op.v
    …

 

Overview

Our Rocq-mechanised proofs are contained in the “formalisation” folder of the artifact. This folder contains an auto-generated Makefile for building and checking our proofs.

All Rocq code is contained in files in the “formalisation/theories” subdirectory of the artifact. The core result is program_soundness in “program.v”, which says that our definition of a program specification implies an Iris-backed notion of soundness for our translational semantics. We also provide a default specification to show that the definition is not trivially vacuous.

Several files of note are listed here:

  - “formalisation/theories/lang.v” - the definition of the syntax for our HeapLang-like source language, as well as the embedding of (runtime) source language terms into (runtime) HeapLang terms
  - “formalisation/theories/assertion.v” - the definition of the syntax for our separation-logic assertion language, as well as the embedding of that language into Iris separation logic propositions
  - “formalisation/theories/translation.v” - the definition of our translation of  source language programs into HeapLang programs, including auto-generation of fresh variables
  - “formalisation/theories/hoare_logic.v” - the definition of our program logic and its encoding into Iris weakest-precondition specifications onHeapLang terms under our embedding
  - “formalisation/theories/program.v” - our definition of whole-program specifications, and our proof that our encoding of our program logic produces sound Iris specifications for the resulting translated HeapLang program
  - “formalisation/theories/adequacy.v” - instantiations of our theory that demonstrate that our definitions are inhabited

Checking proof assumptions

We have marked the most relevant theorems with Rocq commands that will print all assumptions they make. These assumptions will appear in the output when building the proofs as instructed above.

The core soundness result technically has to assume certain user-provided typeclass instances exist. A truly axiomless proof is only possible once the soundness result has been instantiated for a particular program. We therefore also give a small instantiation to show that the definitions we use are not vacuous.

The following three theorems print their assumptions:

 1. program_soundness - this is the core soundness result. Its assumptions are that certain typeclass instances and Iris ghost resources are inhabitable. Its assumption output when checking the proofs is similar to the following:
    
        Section Variables:
        Σ
        : gFunctors
        hlc0
        : has_lc
        encodingG0
        : encodingG A Σ
        H0
        : PartialOrderCompare Σ A AR
        H
        : heapGS_gen hlc0 Σ
        AR
        : relation A
        A
        : Type
 
 2. program_adequate - a proof that a user-provided specification can entail the core soundness result without any additional assumptions. Its assumption output is the following line:
    
        Closed under the global context
 3. default_program_adequate - an example of a specified program, for which soundness holds without any additional assumptions. Its assumption output is the following line:
    
        Closed under the global context

Program logic rules

All the program logic triple rules mentioned in the paper, as well as some typical rules included in such logics (such as consequence, framing) can be found in “hoare_logic.v”. These rules can be visually compared against the paper versions thanks to our notation for assertions and programs.