SLE 2022 Artefact #7 Documentation

Installation

We provide a VM image of a Debian OS. The username and password are both "cogent" (without the quotes). Should root access be needed, the password is also "cogent".

Once logged in, we recommend reviewers to open a terminal. A cogent directory can be found in the home directory, which contains the source code of the artefact.

In order to evaluate the artefact, the following programs are needed. They are already pre-installed so reviewers do not need to manually install anything.

  • The GHC compiler (ghc)
  • The Haskell Cabal (cabal)
  • The Cogent compiler (cogent)

For the purpose of this artefact evaluation, please ignore all the README files contained in the artefact. Reviewers should find all the necessary information in this file, unless we explicitly refer to one.

File Summary

We briefly summarise the directory structure below, so that reviewers are easier to navigate themselves. Please proceed to the [Evaluation Steps] section below for details.

Inside the $HOME/cogent directory, we have the following files and sub-directories that are relevant to this paper.

  • ./Bag.cogent: the program in Figure 2
  • ./cogent/: the Cogent compiler and verification framework, extended with PBT
    • ./src/: the Cogent compiler source code
    • ./src/Cogent/Haskell: the source code for generating Haskell embeddings for PBT
    • ./quickcheck/: the library files for the PBT framework (not to be confused with ./tests-quickcheck, which is irrelevant)
    • ./lib: the Cogent standard library, which includes WordArray
  • ./impl/fs/bilby/: the BilbyFs file system
    • ./quickcheck: Cogent PBT examples presented in the paper
    • ./proof/spec: the functional correctness specification used by BilbyFs
    • ./cogent: the full Cogent source code of BilbyFs

Evaluation Steps

(In this section, we assume all paths are relative to $HOME/cogent, unless specified otherwise.)

The Cogent compiler

The Cogent compiler is extended with a Haskell backend for generating Haskell embeddings for PBT. The implementation of the generator is not detailed in the paper, but reviewers can find the source code in ./cogent/src/Cogent/Haskell/.

The compiler has already been installed in the VM, which will be used for compiling the examples presented in Section 5 and 6. If reviewers want to verify that the source code compiles, it can be done by the following steps:

  • cd ./cogent
  • cabal new-install --installdir=$HOME/.cabal/bin --overwrite-policy=always

But because the source code is identical to when it was last built, the cabal new-install command does not rebuild the compiler and it should report something like "Up to date".

For a general understanding of Cogent, we provide a ./cogent/Bag.cogent file for the program presented in Figure 2, along with a testsuite comprised of small Cogent programs in ./cogent/tests/tests/, and a collection of small examples in ./cogent/examples/.

To run the compiler, the following commands are the most relevant (where <SRC> is the name of the source Cogent program):

> cogent -t <SRC>     # typechecks the <SRC>
> cogent -g -x <SRC>  # generates C code and prints to stdout
> cogent -Q <SRC>     # generates all files needed for PBT
> cogent -h<LEVEL>    # print help messages; <LEVEL> ranges from 0-4: the higher, the more
                      # options and flags are revealed

Auto-completion has been set up in the VM, so that users can hit TAB twice in a terminal after typing cogent to see available commands and flags.

Section 5 and 6: Case studies

The source code for the case studies is located in $HOME/cogent/impl/fs/bilby/quickcheck. We assume $PWD is this directory.

Setup

  1. Set environment variable:
> export COGENT_LIBGUM_DIR=$HOME/cogent/cogent/lib
  1. The PBT library files are located in $HOME/cogent/cogent/quickcheck/. It includes the corres relations and the non-deterministic monad used in Sec 5 and 6.

It can be built with the following command but later steps will build it as a dependency, therefore this step can be skipped.

> cabal new-build cogent-quickcheck-utils

The WordArray example

The code is in the $PWD/wa_example/ directory. The most relevant files are as follows: WordArray.hs defines the specification of wordarray, and also defines the refinement tests. wa.cogent is an instantiation of the wordarray library, which is located in $HOME/cogent/cogent/lib/gum/common/wordarray.cogent, with the foreign C library code defined in $HOME/cogent/cogent/lib/gum/anti/wordarray.ac and $HOME/cogent/cogent/lib/gum/anti/abstract/WordArray.ah.

  1. Generate the C code and binary files:
> cd wa_example
> make
> cd ../

The generated files are put in $PWD/wa_example/build/.

  1. Build and run the tests:
> cabal new-build exe:wa-example-exe --ghc-options="`pwd`/wa_example/build/wa.o"
> cabal new-exec wa-example-exe

NOTE: According to the GHC user guide, GHCi doesn't quite work with foreign functions. Therefore we cannot run the tests interactively in GHCi.

The tests being run can be seen in $PWD/wa_example/Main.hs. We test each property with 100 test cases, which is the default. There are lots of analysis about the tests that can be done with the QuickCheck library (see https://hackage.haskell.org/package/QuickCheck-2.14.2/docs/Test-QuickCheck.html#g:21 but they are out of the scope of this paper).

The readpage example

The files for this example are in $PWD/readpage_example.

Readpage.hs is the top-level file that defines the specification of the program under test and the test driver. Fsop.hs defines and re-exports some definitions that the top-level test file depends on. Readpage_Shallow_Desugar_Tuples.hs is the Haskell embedding of the source Cogent program readpage.cogent. The embedding is pre-generated and manually modified to implement the abstract Cogent types and functions, so it normally shouldn't be re-generated. However, reviewers can use the following command to generate a fresh Haskell embedding and do a diff to see what has been manually modified. The command below generates a file called New_Readpage_Shallow_Desugar_Tuples.hs, which can be compared against Readpage_Shallow_Desugar_Tuples.hs.

> cogent --hs-shallow-desugar-tuples readpage.cogent --flax-take-put -oNew_Readpage

To run the test:

  1. Assume wa-example has been compiled as described above.
  2. cabal new-repl readpage-example gives a GHCi REPL with the Haskell files loaded.
  3. Inside the REPL, *Fsop> Readpage.main (where *Fsop> is the GHCi prompt) runs the test with a preset configuration.
  4. *Fsop> :q to quit the REPL.

To compare the Haskell executable spec with the Isabelle abstract functional correctness spec, reviewers can open the Isabelle theory file in an editor and inspect the specs: Line 522 of $HOME/cogent/impl/fs/bilby/proof/spec/AfsS.thy is the Isabelle spec for readpage, as shown in Figure 5. The actual proof is largely orthogonal to the topic of this paper, so it is not necessary to run Isabelle and spend the time processing the theory files.