This is the artifact accompanying 'Proof Automation for Linearizability in Separation Logic'. The paper describes new proof automation strategies for establishing linearizability, in two ways: by establishing logical atomicity, or contextual refinement. Proof automation strategies for both these goals are built by instantiating Diaframe 2.0, a proof automation framework for Iris's separation logic. (Iris itself is a separation logic framework in Coq.)
The artifact is intended to back the claims made in the paper about the provided proof automation. To do so, the artifact contains the complete source code of Diaframe 2.0, its instantiations for logical atomicity and contextual refinement, and the examples on which the proof automation was benchmarked. The list of claims the artifact backs is as follows:
The artifact contains the source-code of Diaframe 2.0 (diaframe2-artifact-source.tgz) as well as relevant dependencies. It also contains a virtual machine (diaframe2-artifact-vm.ova) where this source-code and dependencies have been pre-installed. Pick a version that suits you, then follow the corresponding instructions below.
diaframe2-artifact-vm.ova)To start the virtual machine, you need virtualization software. We recommend Virtual Machine Manager(tested with version 2.2.1) or VirtualBox. VirtualBox can work with .ova files directly, to use Virtual Machine Manager one needs to convert this to a different format - see the end of this document. For the VM, we recommend at least 4 GB of RAM, although 2GB might also work. If you want to verify files in parallel we recommend 8GB of RAM.
This VM contains all dependencies of Diaframe 2.0, as well as CoqIDE. To check that you have a functional installation, first open CoqIDE (third sidebar icon from the top, or press Start and type 'coq'). Now open (ctrl+O), for example, atomic_spin_lock.v, located in ~/Documents/paper-auto-iris-artifact/diaframe2-artifact-source/diaframe/supplements/diaframe_heap_lang_examples/logatom/. Press Ctrl+End or the 'Go to end' arrow to run the verification. All (non-empty) lines should become green.
To run this verification from the command line, open the terminal and navigate to ~/Documents/paper-auto-iris-artifact/diaframe2-artifact-source/diaframe. First run
eval $(opam env)
to make opam functional. Then run
touch supplements/diaframe_heap_lang_examples/logatom/atomic_spinlock.v && make supplements/diaframe_heap_lang_examples/logatom/atomic_spinlock.vo
to force recompilation/reverification of the spin lock. This should at least print
COQC supplements/diaframe_heap_lang_examples/logatom/atomic_spin_lock.v
and terminate succesfully without printing errors.
The user of the VM image is 'artifact' with password 'artifact'. To change the keyboard layout, press the cog/settings logo, go to 'Region & Language' and add your desired keyboard layout with the '+' button.
diaframe2-artifact-source.tgz)(Tested on 64-bits Ubuntu 20.04, and on a not very recent macbook)
Building Diaframe from source requires a functioning (recent, i.e. >= 2.0.0) version of opam. To check whether this is the case, you can run opam --version. If you do not have a recent version of opam, follow the instructions here.
For macOS, we recommend installing the gtime command with brew install gnu-time.
You can build the artifact by running the following commands:
tar xf diaframe2-artifact-source.tgz
cd diaframe-artifact-source
./build_artifact.sh
This will install all dependencies locally, compile Diaframe, and verify some of the examples listed in the paper. Depending on your system, opam might ask you to install additional system dependencies like libgmp-dev.
Among other things, ./build_artifact.sh creates a local opam switch in the current directory. To make opam able to use this local switch, run
eval $(opam env)
After this, you should be able to compile (parts of) Diaframe. To do so,
first enter the diaframe folder with
cd diaframe
Then, execute the following:
touch supplements/diaframe_heap_lang_examples/logatom/atomic_spinlock.v && make supplements/diaframe_heap_lang_examples/logatom/atomic_spinlock.vo
to force recompilation/reverification of the spin lock. This should at least print
COQC supplements/diaframe_heap_lang_examples/logatom/atomic_spinlock.v
and terminate succesfully without printing errors.
Once you have a functioning opam with the proper dependencies, you can make Diaframe 2.0's various compilation targets. We list some relevant compliation targets here:
diaframe: core of Diaframe 2.0, not instantiated for a specific goaldiaframe-heap-lang: adds proof automation for weakest preconditions and logical atomicity for Iris's default language, HeapLangdiaframe-reloc: adds proof automation for ReLoC's contextual refinementlogatom-voila: compiles all logically atomic examples from Voila's benchmarklogatom-other: compiles all logically atomic examples for the comparison to interactive proofsreloc: compiles all refinement examples to compare to ReLoCTo get a report of compilation times, use make pretty-timed TGTS="{your_target_here}" TIMING_INCLUDE_MEM=0 instead of make {your_target_here}.
The source code of the artifact can be found in the diaframe folder of the source code artifact, at ~/Documents/paper-auto-iris-artifact/diaframe2-artifact-source/diaframe in the virtual machine artifact. This directory contains the following relevant subdirectories:
diaframe contains Diaframe's proof search procedure, not instantiated for a language or goal, and various hint librariesdiaframe_heap_lang contains an instantiation of Diaframe for weakest-preconditions on heap_lang, Iris's default languagetutorial contains five tutorial files with lots of comments, to get users started on using Diaframetests contains some files to test Diaframe's proof search, used for developmentsupplements contains verifications of various examples, and instantiations of Diaframe for ReLoC:supplements/diaframe_heap_lang_examples/comparison contains all examples from the original benchmark of Diaframe 1.0supplements/diaframe_heap_lang_examples/logatom contains all logical atomicity verifications with Diaframe 2.0supplements/diaframe_reloc/ contains the proof automation for ReLoC built with Diaframe 2.0supplements/diaframe_reloc/examples contains all ReLoC examples verified with above proof automation.utils contains various scripts and other utilities, independent of the Coq formalization.For more information on the source code of Diaframe 2.0, see diaframe/README.md.
To get started easily, we have included all Coq dependencies of Diaframe 2.0. These can be found in the coq-stdpp, coq-iris, coq-iris-heap-lang, coq-reloc and coq-autosubst folders, and are not part of the artifact.
If you want to take a shot at using Diaframe 2.0 to verify linearizability of some program, these are the steps you need to take. They are discussed in more detail below.
supplements/diaframe_heap_lang_examplesWe recommend checking out the tutorial-style files first. These can be found in diaframe/tutorial and contain lots of comments to help you get started. It explains the general process of verifying examples, and also explains how to define your own hints. You can also look at the examples in supplements/diaframe_heap_lang_examples/logatom and supplements/diaframe_reloc/examples, although these are not commented on. For getting started on verifying contextual refinement, see tutorial/ex3_xchg_refinements.v. For getting started on verifying logical atomicity, see tutorial/ex4_xchg_logatom.v.
We recommend using any Coq IDE (such as CoqIDE) to interact with Diaframe. Create a new file in a location where Coq can find Diaframe, such as supplements/diaframe_heap_lang_examples.
Get access to heap_lang by adding From iris.heap_lang Require Import proofmode. This makes it possible to write down programs in the heap_lang language, and gives you access to some tactics that may be helpful when verifying heap_lang programs.
Get access to Diaframe 2.0 for heap_lang by adding From diaframe.heap_lang Require Import proof_automation.. This gives you access to the iStepS, iStepsS and iSmash tactics. For more explanation on these tactics, see diaframe/README.md.
In this file, write down your program in the heap_lang language. heap_lang is a lambda-calculus with a heap, deeply embedded in Coq. An example function is:
Definition add_locs : val :=
λ: "l" "r", ! "l" + ! "r".
This defines a Coq-term add_locs, which has type val: a value in the heap_lang language. This value is a function, and when applied to two locations, it returns the sum of their stored values.
add_locs uses the ! operation to load from a heap-location. Note that the binders in the function are strings ("l" and "r"), not regular Coq binders!
Please also see the tutorial files in diaframe/tutorial for more examples and heap operations. You may also wish to consult the heap_lang documentation.
Please see the tutorial files for details and examples on writing specifications for programs. For getting started on verifying contextual refinement, see tutorial/ex3_xchg_refinements.v. For getting started on verifying logical atomicity, see tutorial/ex4_xchg_logatom.v.
After writing the specification, enter the Coq proof mode with Proof. After at least a single whitespace, you can start using Diaframe tactics to construct a proof. Make Diaframe run a chunk of steps with the iStepS tactic. Run the automation until it finishes or gets stuck with the iStepsS tactic. If your goal/invariants contain disjunctions and you wish to use backtracking to choose a side, use the iSmash tactic.
If the function you are verifying is not recursive, proofs usually only consist of Proof. iStepsS. Qed., where one possibly needs to replace iStepsS with iSmash.
It may be the case that Diaframe gets stuck. In this case you can try to manually prove the problematic part of the goal, then resume Diaframe's automation with iStepsS. Another option is to add appropriate hints, thereby teaching Diaframe to handle the problematic parts of the goal. See the tutorial files in diaframe/tutorial for examples.
Please see diaframe/README.md for some pointers on extending or changing Diaframe 2.0. This file also contains more information on the directory structure and the function of each of the Coq source code files.
To evaluate the artifact, we encourage you to check that the artifact supports the following list of claims made in the paper:
Each claim above comes with a subsection below, describing our suggestions to verify the claims.
The example contextual refinement from §2 can be found in supplements/diaframe_reloc/examples/paper_example.v.
To establish that the proof automation is functional, verify this file using either a Coq IDE, or the command line.
When using the command line, the procedure is:
diaframe folder (not diaframe/diaframe!).eval $(opam env)touch supplements/diaframe_reloc/examples/paper_example.v && make supplements/diaframe_reloc/examples/paper_example.vo. This will print some things to the console: the two assumed Axioms under which the proven refinement holds. These axioms all stem from dependencies of Diaframe 2.0. The first one is an admitted lemma in ReLoC, whose fix is yet to be merged. The second one is functional extensionality, which is assumed by autosubst, a dependency of ReLoC.To check that the proof automation works for other examples, see Claim 6: §5.4: The comparison to interactive proofs in ReLoC is accurate (Fig. 9).
One can check that the proof automation follows the procedure from §2.3 by 'stepping' through the proof. That is, replace calls to the iStepsS. tactic with multiple iStepS. and verify that each new goal is what you expect from §2.3.
The rules in Figure 4 are proven in the separate file supplements/diaframe_reloc/correspondence_figure_4.v. This file only demonstrates that the shown rules hold, and is not used by the proof automation itself. The corresponding rules in the proof automation are more abstract and their correspondence is harder to see. For completeness, we list them below.
refines_values_abduct in supplements/diaframe_reloc/proof_automation.v, line 17-20 for both val_Z and val_fun.refinement_bind in supplements/diaframe_reloc/proof_automation.v, line 102-106 for reloc_apply. This includes machinery to find appropriate evaluation contexts K automatically.left_execute_reduction_compat in supplements/diaframe_reloc/symb_exec.v, line 244-246 contains the proof of exec_l in more generality. Its effect is that any specification for WPs can be used as a specification for a refinement, but this is quite difficult to see directly.left_execute_reduction_compat in supplements/diaframe_reloc/symb_exec.v, line 244-246 contains the proof of exec_l in more generality. Its effect is that any specification for WPs can be used as a specification for a refinement, but this is quite difficult to see directly.right_execute_reduction_compat in supplements/diaframe_reloc/symb_exec.v, line 96-98 contains the proof of exec_r in more generality. Its effect is like above, but for thread pool specifications.AU-ACCESS-DIAFRAME holds.The example logically atomic triple from §3 can be found in supplements/diaframe_reloc/examples/paper_example.v.
To establish that the proof automation is functional, verify this file using either a Coq IDE, or the command line.
When using the command line, the procedure is:
diaframe folder (not diaframe/diaframe!).eval $(opam env)touch supplements/diaframe_reloc/examples/paper_example.v && make supplements/diaframe_reloc/examples/paper_example.voNote that above file contains both the contextual refinement and the logically atomic triple. The logical atomicity part starts at line 65.
To check that the proof automation works for other examples, see Claim 4 and Claim 5.
One can check that the proof automation follows the procedure from §3.3 by 'stepping' through the proof. That is, replace calls to the iStepsS. tactic with multiple iStepS. and verify that each new goal is what you expect from §3.3.
The AU-ACCESS-DIAFRAME rule is called atomic_update_access and located in diaframe/lib/atomic.v at line 113-121. It states that an atomic_update entails the desired mask shift, although generalized somewhat. Unlike in the paper, the quantification variable v is a telescope (dependent list of variables) in the formalization. To avoid universe problems, we do not use an option type, and instead require that we can establish default inhabitants of the quantified type (Inhabited TB). Morally, the rule is precisely as described in the paper. There is an additional generalization on masks, since the atomic_updates in the formalization have an additional 'inner mask' parameter Ei, and an established atomic_update for some (outer) mask Eo can be used for any larger mask E.
Abduction hints (§4.1) are defined in diaframe/solve_defs.v, line 113, transformer hints are defined in lines 162-166. The abduction hint definition in the artifact is also parametrized by a modality and existential quantification, although this feature is not used. The hypothesis transformer hint definition TransformHyp takes an additional boolean p to indicate whether the hypothesis is from the intuitionistic context.
Notations for these hints are also defined in diaframe/solve_defs.v, from line 409-594. The notation is slightly different between paper and artifact:
We have included a tutorial file to demonstrate building new proof automation with Diaframe 2.0. It is located in tutorial/ex5_custom_proof_automation.v, and builds some proof automation for WP from scratch. Following this file should allow you to get a feel for the behavior of the proof automation, and how one can extend and change it by adding hints.
The following steps will be quite technical and require experience and knowledge of Coq's typeclasses and Iris. We recommend checking just the behavior of the hints and strategies above, if one is less familiar with the setting.
The abduct-wp-val abduction hint is collect_modal_wp_value in diaframe/symb_exec/weakestpre.v, line 542-545.
Both abduct-wp-bind and abduct-sym-ex-logatom are consequences of abduct_from_execution in diaframe/symb_exec/weakestpre.v, line 534-539. The abduct_from_execution abduction hint is quite abstract, so this is hard to see directly. The important part is on line 536, where (given some WP e {{ Φ }} as input), the hint tries to find e = K[e_in'], where we have any kind of specification for e_in' -- a ReductionTemplateStep. For abduct-wp-bind, this will be reduction_step_from in line 307-309. For abduct-sym-ex-logatom, this will be an instance of AtomicReductionStep', defined in diaframe/symb_exec/weakestpre_logatom.v.
Finally, the recursive hint rules like abduct-wand are found in diaframe/hint_search/lemmas_abd.v. abduct_wand can be found on line 144-149.
The au-intro-pre abduction hint is abduct_aupd_as_gfp in diaframe/lib/atomic.v, line 69. Note AU_pre is defined differently in the artifact, namely as run_greatest_fixpoint (λ Ψ, atomic_acc' Eo Ei α Ψ β Φ). This reflects the definition of atomic updates in Iris. The transformer hint which corresponds to au-intro-go is gfp_introduce_all_laterable in diaframe/lib/greatest_fixpoint.v.
The implementation of this strategy resides in the diaframe/steps folder. The strategy in 4.4 is a simplified representation, the implementation contains more details and cases. We list the correspondence of the top-level cases to the files in diaframe/steps:
∀ x. G is handled in introduce_var.vU -∗ G is handled in introduce_hyp.vA is handled in solve_one.v|={E1, E2}=> ∃ x. L ∗ G is handled in solve_sep.vT is handeld in transform.vTo (re)verify all the files for the Voila comparison, navigate to the diaframe folder and run make clean and rm time-of-build*. Make sure your opam switch is functional with eval $(opam env). Then run the following command:
make diaframe diaframe-heap-lang && make pretty-timed TGTS="logatom-voila" TIMING_INCLUDE_MEM=0
to verify all of the examples listed in Fig. 7. On a single core, this should take about 5 minutes, depending on your hardware. Enable parallel verification by additionally supplying make with the -j NUMBEROFJOBS argument. Note that due to multi-threading, this will probably give slower times for each file.
Once this is finished, make should print a table of verification times for each file.
You may also wish to check that the contents of each supplements/diaframe_heap_lang_examples/logatom/atomic_* file correspond to real verifications of logically atomic triples, the same as those performed by Voila.
Each row in figure 7 corresponds to a file in supplements/diaframe_heap_lang_examples/logatom/. Usually, the file name is atomic_{name}.v, where you should replace {name} with the name entry in the figure.
The number for total is just the total lines in the file. The time entry refers to compilation/verification time, which can be obtained with above make call. The impl entry is obtained by counting all the lines that define the functions (: val) which we will verify. For example, for atomic_ticketlock.v this is line 24-41. The proof entry refers to any lines enclosed between Proof. and Qed.. For atomic_ticketlock.v, this is line 47-50.
We have not run Voila on new benchmarks. The numbers for Voila were found by inspecting the source code, which is available here
here.
The number for total is just the total lines in the file. The number for proof is a bit harder to construct. We went manually through each file, counting lines which constituted proof burden for the user. By 'constitute proof burden', we mean:
inhale/exhale/fold statementsmake_atomic using ... {, which explicitly instructs Voila to prove the enclosed statement atomicopen_region using which explictly instructs Voila to open a region (akin to an invariant)update_region using which explictly instructs Voila to update a region (akin to an invariant)assert/assume statementslemmas, and explicit use of such lemmasThe table generated with the artifact and Table 1 in the submitted version of the paper differ in various places. We mention and comment on large differences below.
bag_stack: larger annotation and proof burden. We added an additional is_list_skel predicate and corresponding hints, making it longer, but this makes the proof (in our opinion) more elegant and faster to verify. This increased the total line count with 21 lines, and the proof burden with 10 lines. The old version of bag_stack still works, but takes about twice as long to verify.barrier_client: smaller annotation and proof burden, due to fixes and improvements in Diaframe's implementationlclist: smaller annotation and proof burden, due to fixes and improvements in Diaframe's implementationmsc_queue: smaller annotation and proof burden, due to fixes and improvements in Diaframe's implementation, and a small change to the used ghost statequeue: smaller annotation and proof burden, due to fixes and improvements in Diaframe's implementation, and a small change to the used ghost staterwlock_duolock: smaller annotation and proof burden, due to fixes and improvements in Diaframe's implementationtotal: The total numbers have changed, not just because of the changed numbers. The calculation of totals also turned out to be wrong in the submitted version (you can check this by manually adding, for example, all implementation line counts - the actual total does not match the listed total)An updated screenshot of Table 1 is included in this artifact as table1_updated.png. The takeaway from this updated table is that the actual numbers are slightly better than originally presented in the paper: averaged over all examples, Diaframe actually requires about 0.4 lines of proof work per line of implementation (321 lines of proof work for 823 lines of implementation), whereas in the paper we spoke of 0.5 lines of proof work per line of implementation.
The files for Fig. 8 also reside in supplements/diaframe_heap_lang_examples/logatom/.
To (re)verify all the files for this comparison, navigate to the diaframe folder and run make clean and rm time-of-build*. Make sure your opam switch is functional with eval $(opam env). Then run the following command:
make diaframe diaframe-heap-lang && make pretty-timed TGTS="logatom-other" TIMING_INCLUDE_MEM=0
to verify all of the examples listed in Fig. 8. On a single core, this should take about 20 minutes, depending on your hardware. Enable parallel verification by additionally supplying make with the -j NUMBEROFJOBS argument. Note that due to multi-threading, this will probably give slower times for each file.
Once this is finished, make should print a table of verification times for each file.
You may also wish to check that the contents of each file correspond to real verifications of logically atomic triples.
Each row in figure 8 corresponds to a file in supplements/diaframe_heap_lang_examples/logatom/. Usually, the file name is {name}.v, where you should replace {name} with the name entry in the figure.
The number for total is just the total lines in the file. The time entry refers to compilation/verification time, which can be obtained with above make call. The impl entry is obtained by counting all the lines that define the functions (: val) which we will verify. For example, for msc_queue_faithful.v this is line 7-58. The proof entry refers to any lines enclosed between Proof. and Qed..
The numbers for the Michael Scott queue msc_queue are obtained by adding the results of multiple files: msc_queue_faithful.v and queue_lib.v. The queue_lib.v file defines some shared lemmas and hints for verifying simpler queues, which are also included in the artifact: queue.v and msc_queue_simplified.v. These queues do not need prophecy variables in their verification.
Both the RDCSS example aswell as the Elimination stack example were taken from the Iris examples repository: rdcss and elimination_stack.
The number for total is just the total lines in the file. The proof entry refers to any lines enclosed between Proof. and Qed..
The files for Fig. 9 reside in supplements/diaframe_reloc/examples/.
To (re)verify all the files for this comparison, navigate to the diaframe folder.
Make sure your opam switch is functional with eval $(opam env). We advise you not to clean if you have verified the msc_queue_faithful.v from Claim 5: this file will be required for this claim, and takes a while to recompile.
Run the following command:
make diaframe-reloc && make pretty-timed TGTS="reloc" TIMING_INCLUDE_MEM=0
to verify all of the examples listed in Fig. 8. On a single core, this should take about 10 minutes, depending on your hardware and assuming msc_queue_faithful.v has been compiled. Enable parallel verification by additionally supplying make with the -j NUMBEROFJOBS argument. Note that due to multi-threading, this will probably give slower times for each file.
The ReLoC source code is included in the artifact: the coq-reloc folder in diaframe2-artifact-source. The examples are located at coq-reloc/theories/examples/. The number for interactive total is just the total lines in the file. The interactive proof entry refers to any lines enclosed between Proof. and Qed. in these files.
Unlike the logical atomicity examples, we directly import and reverify the compiled examples of ReLoC. This is mainly for convenience, since we are importing ReLoC anyway. This makes calculating the line numbers a bit different from the previous examples.
The impl entry is obtained by counting all the lines in the corresponding ReLoC example file that define the functions (: val) which we will verify. The proof entry is obtained by counting any lines enclosed between Proof. and Qed. in the Diaframe files located at supplements/diaframe_reloc/examples/. The total entry is then derived as {interactive total} - {interactive proof} + {proof}: if one would reverify from scratch, the longer interactive proofs would be replaced by the shorter more automated proofs.
In §5.3, the paper claims that logical atomicity is stronger and implies contextual refinement. We have worked out this fact for the Michael Scott queue, in the supplements/diaframe_reloc/exampes/msc_queue_refinement.v file. This imports the logically atomic specification of the Michael Scott queue, then directly proves contextual refinement using this fact.
Specifically, it proves the Michael Scott queue contextually refines a coarse-grained queue implementation, with lemma queue_refinement in line 94. The coarse-grained queue CG_queue is fully defined in the current file. MS_queue_original is built with the new_queue, dequeue and enqueue methods imported from msc_queue_faithful.v
If you want to use virt-manager (virsh and friends), you have to convert the diaframe2-artifact-vm.ova file to a .qcow2 file. You can do this as follows. An .ova file is just a tar archive, extract it:
tar xvf diaframe2-artifact-vm.ova
This will print a list of extracted files (in the current directory). The .vmdk file can be converted to a .qcow2 file, which is the format virt-manager prefers.
Now create a new virtual machine, choose 'import existing disk image' and select the .qcow2 file. You should now be able to run the artifacts virtual machine.
make: *** no rule to make target.Ensure that you have run eval $(opam env). If you haven't, opam switch list will warn you that your environment is not in sync with the current switch. If just running eval $(opam env) does not fix this, try first running make clean before retrying.
Nothing to be done for ..., while I have just touched the sourceSee above.
./build_artifact.sh complains: no compiler matching 4.14.0 found.Run opam update to get an up to date list of OCaml compiler versions.