We provide a VM image of a Debian OS. The username and password are both "cogent" (without the quotes). Should root access be needed, the password is also "cogent".
Once logged in, we recommend reviewers to open a terminal.
A cogent
directory can be found in the home directory, which contains
the source code of the artefact.
In order to evaluate the artefact, the following programs are needed. They are already pre-installed so reviewers do not need to manually install anything.
For the purpose of this artefact evaluation, please ignore all the README files contained in the artefact. Reviewers should find all the necessary information in this file, unless we explicitly refer to one.
We briefly summarise the directory structure below, so that reviewers are easier to navigate themselves. Please proceed to the [Evaluation Steps] section below for details.
Inside the $HOME/cogent
directory, we have the following files and
sub-directories that are relevant to this paper.
./Bag.cogent
: the program in Figure 2./cogent/
: the Cogent compiler and verification framework, extended with PBT./src/
: the Cogent compiler source code./src/Cogent/Haskell
: the source code for generating Haskell embeddings for PBT./quickcheck/
: the library files for the PBT framework (not to be confused with ./tests-quickcheck
, which is irrelevant)./lib
: the Cogent standard library, which includes WordArray./impl/fs/bilby/
: the BilbyFs file system./quickcheck
: Cogent PBT examples presented in the paper./proof/spec
: the functional correctness specification used by BilbyFs./cogent
: the full Cogent source code of BilbyFs(In this section, we assume all paths are relative to $HOME/cogent
, unless specified otherwise.)
The Cogent compiler is extended with a Haskell backend for generating Haskell embeddings
for PBT. The implementation of the generator is not detailed in the paper, but reviewers
can find the source code in ./cogent/src/Cogent/Haskell/
.
The compiler has already been installed in the VM, which will be used for compiling the examples presented in Section 5 and 6. If reviewers want to verify that the source code compiles, it can be done by the following steps:
cd ./cogent
cabal new-install --installdir=$HOME/.cabal/bin --overwrite-policy=always
But because the source code is identical to when it was last built, the cabal new-install
command does not rebuild the compiler and it should report something like "Up to date".
For a general understanding of Cogent, we provide a ./cogent/Bag.cogent
file for
the program presented in Figure 2, along with a testsuite comprised of small
Cogent programs in ./cogent/tests/tests/
, and a collection of small examples
in ./cogent/examples/
.
To run the compiler, the following commands are the most relevant (where <SRC> is the name of the source Cogent program):
> cogent -t <SRC> # typechecks the <SRC>
> cogent -g -x <SRC> # generates C code and prints to stdout
> cogent -Q <SRC> # generates all files needed for PBT
> cogent -h<LEVEL> # print help messages; <LEVEL> ranges from 0-4: the higher, the more
# options and flags are revealed
Auto-completion has been set up in the VM, so that users can hit TAB twice in a
terminal after typing cogent
to see available commands and flags.
The source code for the case studies is located in $HOME/cogent/impl/fs/bilby/quickcheck
.
We assume $PWD
is this directory.
> export COGENT_LIBGUM_DIR=$HOME/cogent/cogent/lib
$HOME/cogent/cogent/quickcheck/
. It includes
the corres relations and the non-deterministic monad used in Sec 5 and 6.It can be built with the following command but later steps will build it as a dependency, therefore this step can be skipped.
> cabal new-build cogent-quickcheck-utils
WordArray
exampleThe code is in the $PWD/wa_example/
directory. The most relevant files are as follows:
WordArray.hs
defines the specification of wordarray, and also defines the
refinement tests. wa.cogent
is an instantiation of the wordarray library, which
is located in
$HOME/cogent/cogent/lib/gum/common/wordarray.cogent
, with the foreign C
library code defined in $HOME/cogent/cogent/lib/gum/anti/wordarray.ac
and
$HOME/cogent/cogent/lib/gum/anti/abstract/WordArray.ah
.
> cd wa_example
> make
> cd ../
The generated files are put in $PWD/wa_example/build/
.
> cabal new-build exe:wa-example-exe --ghc-options="`pwd`/wa_example/build/wa.o"
> cabal new-exec wa-example-exe
NOTE: According to the GHC user guide, GHCi doesn't quite work with foreign functions. Therefore we cannot run the tests interactively in GHCi.
The tests being run can be seen in $PWD/wa_example/Main.hs
. We test each property with
100 test cases, which is the default. There are lots of analysis about the tests that can
be done with the QuickCheck library (see https://hackage.haskell.org/package/QuickCheck-2.14.2/docs/Test-QuickCheck.html#g:21 but they are out of the scope of this paper).
readpage
exampleThe files for this example are in $PWD/readpage_example
.
Readpage.hs
is the top-level file that defines the specification of the
program under test and the test driver. Fsop.hs
defines and re-exports some
definitions that the top-level test file depends on.
Readpage_Shallow_Desugar_Tuples.hs
is the Haskell embedding of the source
Cogent program readpage.cogent
. The embedding is pre-generated and manually
modified to implement the abstract Cogent types and functions, so it normally
shouldn't be re-generated. However, reviewers can use the following command
to generate a fresh Haskell embedding and do a diff
to see what has been
manually modified. The command below generates a file called
New_Readpage_Shallow_Desugar_Tuples.hs
, which can be compared against
Readpage_Shallow_Desugar_Tuples.hs
.
> cogent --hs-shallow-desugar-tuples readpage.cogent --flax-take-put -oNew_Readpage
To run the test:
wa-example
has been compiled as described above.cabal new-repl readpage-example
gives a GHCi REPL with the Haskell files loaded.*Fsop> Readpage.main
(where *Fsop>
is the GHCi prompt) runs the test with a preset configuration.*Fsop> :q
to quit the REPL.To compare the Haskell executable spec with the Isabelle abstract functional
correctness spec, reviewers can open the Isabelle theory file in an editor and
inspect the specs: Line 522 of $HOME/cogent/impl/fs/bilby/proof/spec/AfsS.thy
is the Isabelle spec for readpage
, as shown in Figure 5. The actual proof is
largely orthogonal to the topic of this paper, so it is not necessary to run
Isabelle and spend the time processing the theory files.