# Specifying Concurrent Programs in Separation Logic: Morphisms and Simulations. This document provides the overview for the OOPSLA 2019 artefact: _Specifying Concurrent Programs in Separation Logic: Morphisms and Simulations_, a companion to the paper conditionally accepted to OOPSLA 2019 under the same name. The artefact contains all the `Coq` sources of the developments presented in the submission, including the meta-theory of FCSL, and the example case studies. It also contains additional examples that the main body submission does not discuss. Some, such as readers/writers locks, are presented in the Extended Technical Report, which has also been included in the artefact. ## Getting Started Guide We provide a VM image in Open Virtual Appliance (OVA) format, FCSL-oopsla2019.ova, which can evaluate the FCSL developments out of the box. An OVA is an Open Virtualization Format (OVF) file packaged together with all of its supporting files (disk images, etc.). Any virtualization software compatible with the OVA format should be sufficient to work with the provided image. The virtual machine image we provide was created using Oracle Virtual Box Version 5.2.30 r130521 (with the corresponding Extension Pack). Both can be downloaded free of charge from , and GNU/Linux, macOS, and Windows versions are supported. ### FCSL-oopsla2019.ova **OVA Contents** The virtual machine comes with the following components pre-installed: - Ubuntu 18.04 LTS - OPAM 2.0.4, - OCAML 4.07.1, - Coq 8.10+beta2, - Mathcomp 1.9.0 (SSReflect's Mathematical Components Library), - GNU Emacs 26.2 + ProofGeneral (MELPA: 20190618.1328), - CoqIDE 8.10+beta2 The VM is more-over set up with 4GB of RAM, and a 16GB (extensible) virtual hard drive. This settings can be changed using the VM manager to optimize evaluation, but we suggest not reducing the virtualized RAM setting. **Booting Up** Import the provided VM image into VirtualBox (or other virtualization software supporting the `.ova` standard) and boot it up. It should start and log in automatically with the `fcsl` user. **Credentials** The `fcsl` user's password is also `fcsl`, and `sudo` commands will accept `fcsl` the same password. ### Quick Evaluation The source code of the FCSL project (located at `~/fcsl-oopsla2019`) has already been compiled. CoqIDE and Emacs (with ProofGeneral) can be used to inspect and navigate the fcsl source files in the `~/fcsl-oopsla19` folder. Both can be invoked from the terminal, or, alternatively, from the links in the side-bar. Alternatively, to re-compile the sources it suffices to open the terminal, and once in the `~/fcsl-oopsla19` directory, build the sources invoking GNU make. ```shell make clean; make -j N ``` where N is the desired number of parallel jobs (use `-j` without providing a specific `N`to run with the maximal, architecture-supported number of parallel jobs). The build process output should not contain neither errors nor warnings when running out of the box. Note that there may be warnings if other unsupported Coq versions are used, .e.g. the nightly builds from the development versions of Coq and Mathcomp provided by the `dev` versions in `opam`. The building process usually takes from 5 to 10 minutes. ### Alternative Manual Installation Should the reviewers want to evaluate the sources outside the VM sandbox, we provide detailed installation instructions in the `INSTALL.md` file in `~/fcsl-oopsla19`. ### Further Documentation In addition to this Overview, and the first-round submission `oopsla19-paper193.pdf`, the `~/fcsl-oopsla19/` folder also provides an Extended Technical Report, `oopsla19-paper193-TR.pdf`. ## Step-by-Step Instructions In this section, we provide a more in-depth description of the artefact, describe how to navigate and evaluate the Coq implementation, and tie concepts and examples presented in the manuscript (and in the TR) to their respective implementations. ### Detailed Evaluation The artefact supports the developments described in the paper in both a theoretical and practical way. First, it provides a complete bottom-up mechanization of FCSL in Coq. We define (and, prove properties of): the notions of PCMs, their morphisms; the subjective states, transitions and resources; resource morphisms and simulations; the denotational semantics of Hoare triples as a shallow embedding, the inference rules of the program logic and its soundness proof; and several lemmas for manipulating proof-outlines. In this way, the artefact implements the formalization of all the concepts defined throughout the paper, and also it constitutes a proof of the soundness of the whole FCSL in itself. Moreover, it also implements a practical, mechanized specification and verification framework for FCSL, which allows us to prove the specification of all the examples in the paper (and others, described below). #### Type-Checking, Proof-Checking and Consistency **Type Checking is Proof Checking.** Since we are using a dependently typed theory to mechanize our development, proof scripts are checked by Coq's compiler `coqc`. All the claims in the paper, from the definition of f-simulations, to the meaning of FCSL triples and resources, and the specification of client programs reduce to asserting that a certain definition has a valid Coq Type. As we mentioned before, the build script passes on all files. **No Admitted Proofs.** Our development does not contain `Admitted` proofs. All the developments presented in the paper are completely mechanized, without unfinished proofs, or hidden meta-theory assumptions. **Axioms.** We do rely though, on two _standard_ axioms on top of Coq: propositional and functional extensionality, which are known to be compatible with Coq's underlying type-theory. These are introduced in `pcm/axioms.v`. #### PCM Morphisms and Resource Morphisms The algebra of PCM morphisms is given in the `pcm/morphism.v` file. Moreover, the definition of PCM and various PCM constructions, including product PCMs and sub-PCMs are given in the `pcm/pcm.v` file. The former file also contains the definitions of Compatibility relations, and constructors for those, e.g., kernels, equalizers. The definition of a *resource morphism* is given in `system/world.v` file (see `Module Morph`). _A potential source of confusion: *PCM morphisms* and *Resource morphisms* are different notions!_ #### A Minimalist Program logic. In the paper, we claim that morphisms allow us to implement a minimalist program logic with just the 9 inference rules introduced in Section 3.5. These are formalized in `system/model.v`. There, the definition of the `vrf` predicate (Appendix D.3) and the `ST` type are made opaque, so it can be verified that we **only** export the described set of rules. ##### Denotational Semantics of Triples and Soundness of FCSL. Appendix D presents the denotational semantic of FCSL's Hoare types, following the approach of the previous FCSL implementation and reconciled to allow for resource morphisms. The concepts and lemmas presented in D.1 and D.2 are mechanized in `system/process.v`. Then, the predicate transformers `always` and `after`, and their properties, are implemented in `system/always.v`. Finally, Theorem D.14 sketches the statement of the soundness of FCSL, decomposing the proof in three items. The proofs of all three of them, are developed in `system/model.v`. - All the inference rules (described in Section 3 using the `vrf` predicate) are valid. - The `ST V A` and `{P} A {Q} @ V` types are complete lattices, so they admit a fix point by the Knaster-Tarski theorem (`system/domains.v`). - Program constructors (Section D.1) are monotone. #### Structuring and Composing Incremental Lock Functionality. Throughout Sections 2 and 4, we describe how morphisms allow to decompose and recombine resource functionality in order to build a classic example, CSL style spin-locks. These developments are mechanized in `examples/lockoopsla.v`. #### Indexed families of Morphisms and Quiescence Section 5 of the paper describes the use of indexed families of morphisms to implement _quiescence_. The `examples/treiber.v` file contains an implementation of Treiber's non-blocking stack, which implements the erasing of ghost histories described in Section 5. Moreover, the file `examples/span.v` applies quiescence to the prove of a concurrent graph spanning-tree algorithm. ### Project Structure We list here the contents of the artefact, and tie the foremost concepts in the paper and the Extended Technical Report (TR) to the files which implement them. More detailed explanations are provided as comments on each file. #### `pcm/` directory Contains the theory of PCMs and their morphisms. Implements the different PCMs used throughout the paper, e.g. the `natmaps` used to implement histories, or `mutex` PCM used for exclusive ownership. | File name | Description | | :----------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------- | | pcm.v | Implementation of partial commutative monoids (PCMs). | | morphism.v | PCM morphisms and their compatibility relations (Appendix B). | | lift.v | Lifting (total) commutative monoids to PCMs. | | unionmap.v | Reference implementation of PCM of finite maps under disjoint union. | | automap.v | Some automation in the form of overloaded lemmas, for reasoning about finite maps under commutativity and associativity of disjoint union. | | heap.v | Refined implementation of a heaps. | | natmap.v | Finite maps with nat domain. Includes reasoning about last key in the domain, fresh key unused in the domain, etc. The basis of the type of histories. | | mutex.v | A generalized ownership PCM and its instances. | | notation.v | Notation for accessing the ghosts of morphed resources, and associated rewriting lemmas. | | prelude.v | Preliminaries. | | seqperm.v | A theory of permutations over non-equality types. | | pred.v | Support for set-based notion; i.e., writing x \In X. | | axioms.v | Axioms: propositional and functional extensionality. | #### `finmap/` directory Contains the theory of finite maps with an ordered-type domain. | File name | Description | | :---------- | :-------------------------------- | | ordtype.v | Implementation of ordered types | | finmap.v | Theory of finite maps | #### `system/` directory Contains the implementation of FCSL as a shallow embedding in Coq. | File name | Description | | :----------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | domain.v | Basic domain theory: partially-ordered sets, complete lattices, CPOs, monotonicity, continuity, Knaster-Tarski and Kleene theorems. Provides the foundation for the soundness of the `fix` rule (Appendix D.3). | | state.v | Definition of subjective states, pairing, projection, and transposition (Section 3.1) | | world.v | Definition of resources, transitions (Section 3), and some constructions over them: e.g., state product, transition coupling (Appendix A), resource restriction (Appendix B). | | tensor.v | Definition of the tensor combination of resources (Section 2). | | schedule.v | Data type of paths (schedules). Paths select a position in a term where to reduce. | | process.v | Definition of action trees (Appendix D.1). | | always.v | Definition of the predicate transformers `always`, and `after` for adherence to a resource (Appendix D.3). | | model.v | Implementation of the denotational semantics of programs as sets of trees, the `vrf` predicate transformer, the inference rules and the lemmas for reasoning about Hoare triples (from Section 3 of the paper, and Appendix D). | #### `examples/` directory The `examples` folders list the case studies. The main developments presented throughout the paper are given in `examples/lockoopsla.v`. | File name | Description | | :------------ | :-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | priv.v | Implementation of a resources of: private heaps Priv, shared heaps Shar, and a simple concurrent allocator. | | span.v | A concurrent version of a DFS-style in-place computation of a graph spanning tree (full functional correctness verified). | | treiber.v | Implementation of Treiber's non-blocking stack. Implements quiescent clients. | | lockoopsla.v | Locks: A decomposed implementation of CSL lock, as described in the current submission. | | lock.v | Locks: A different decomposed implementation of a CSL lock (from the Technical Report). A decomposed implementation of a Readers/Writers lock, explained in the same document.| | ticket.v | Ticketed Locks. | | flatcomb.v | Implementation of flat combiner. | | stackcomb.v | Instantiating flatcomb with stacks; also coupling flatcomb with another resource, to simultaneously execute the transitions of that resource with the transitions for helping of the flat combiner. | #### Project root directory (`./`) Contains build system files, auxiliary scripts and docs | File name | Description | | :--------------------- | :------------------------------------------------------------------------------------------------------------------------------------------- | | Makefile | Makefile file for building the project using `coq_makefile` utility | | \_CoqProject | List of the project's Coq files (`coq_makefile` generates Makefiles from it, also some IDE-like and auxiliary tools recognize \_CoqProject | | dependency\_graph.sh | Script to generate a graphical representation of the library dependencies (see the script to check its dependencies) | | options.v | Global project's Coq options | #### Code Navigation and Highlights The names in the source code *mostly* correspond to the names in the paper (modulo LaTeX-style representation of Greek letters and similar). ##### Some notation/name correspondence | Notation on paper | Notation in source code | Name on paper | Name in source code | Note | | :---------------: | :---------------------: | :------------ | :------------------ | :--- | | • | `\+` | join | `join` | | | ◁ | `\<` | self-framing | `zig` | | | ▷ | `\>` | other-framing | `zag` | | | ⇒ | `\->` | singleton finite map (from nats) | `nm_pts` | | | | `(idx).` | | | This means we keep the projection after the dot (`.`) intact | | ⊥ | | compatibility relation | `orth` | | | | | partial functional | `funcP` | | | | | other-fixity | `otherP` | | | | | locality | `zagzigP` | | | | | invariance of Sigma | `coherentP` | | | [Γ]. {P} A {Q}@V | `{Γ}, ST [V] (P, Q)` | Hoare type | The type of programs inhabiting a resource V, returning values of type A, with pre-condition P and post-condition Q. Γ binds logical variables in P and Q. | ##### Mapping Formal Concepts We map some of the important concepts and formal structures (formally defined in Section 3) to their implementation constructors and notations. | Name or Convention | Meaning | Note | | :---------------- | :------- | :--- | | `CohPx2` | Constructor of a *resource invariant* | a definition which uses this constructor ends in `coh` | | `TransPx2` | Constructor of a *transition* | a definition which uses this constructor ends in `_tr` | | `ResPx` | Constructor of a *resource* | | | `State` | *state* constructor | | | `[morphism of ...]` | PCM morphism constructor | | | `[pcm of ...]` | PCM constructor | e.g. `[pcm of A * B]` is a product PCM constructor | #### Other Examples We implement other case studies, briefly mentioned in the paper. Some of them are described in further detail in the TR. ##### Locks In the TR, we present a different approach to implement locks using resource morphisms, and present resources for different lock implementations and disciplines: spin locks, ticketing, and readers/writers locks. The `examples/lock.v` file contains these developments, structured as follows. - `Spin`. The Module `Spin` is the implementation of *spin-locks* and verification against a generic spec in terms of histories. - `CSLX`. The Module `CSLX` contains the implementation of a resource for heap ownership transfer specific to exclusive locking. - `CSL`. The Module `CSL` is a the final combination of `Spin` + `CSLX` into the module for exclusive locking, with the lifting of the `genlock` and `genunlock` methods from `Spin` to `exlock` and `exunlock` in `CSL`. - `RWX`. The Module `RWX` is the implementation of a resource for heap ownership transfer and reader counting, specific to *reader-writer* locking. - `RW`. The Module `RW` is the final combination of `Spin` (writer) + `RWX` + `Spin` (reader) into the module RW for reader-writer locking, with the lifting and combination of genlock and genunlock methods from `Spin` (writer) and `Spin` (reader) to a number of methods for reader-writer locking, e.g. `wr_lock`, `wr_unlock`, `prologue`, `epilogue`, and `read`. ###### Treiber Stack - `examples/treiber.v` presents the re-implementation using morphisms of the previous FCSL resource and specification for a Treiber stack. Here, we use morphisms to combine the internal Treiber resource with the concurrent allocator Priv, and also to implement _quiescence_, in the spirit of the example in Section 5. ###### Flat Combiner - `examples/flatcomb.v` presents the re-implementation of previous FCSL developments which implement _helping_. The generic flat combiner resource, FCI(X, M) takes two parameters X and M, where X is the ghost state PCM—e.g. it can be instantiated with histories or natural numbers—, and M specifies the helping interface, that is the methods that helper threads execute. The exported API consists of the methods fc_request and fc_collect, used for respectively requesting and collecting help, and the fc_help method by which helper threads perform their tasks, as specified by M. We use morphisms to obtain a stateful flat combiner FC(X, M) as a combination of FCI(X, M) and Priv. - `examples/stackcomb.v` presents some clients of the flat combiner. In the first one, we instantiate X in FC(X, M) with the PCM of histories, and M with a complete stack API (i.e. `push` and `pop` methods) to implement a flat combining Stack. The API of the latter, consisting of the `fc_push` and `fc_pop` methods, provides a stack whose push and pop methods are implemented via helping. In the second client, we instantiate X in FC(X, M) with the PCM of natural numbers, and again M with a stack API. The resulting resource, StackN, combines the stack instance of the flat combiner with a simple counter method where the collect_tr transition of the flat combiner is coupled with the increment transition of the counter. As a result, the final client (obtained through _morphing_) counts calls to `fc_push`. As the `collect_tr` transition is the linearization point of the flat combiner helping functionality, the coupling achieves synchronicity: whenever the flat combiner helps to push or pop an element off the stack, the counter, which is in a separate object, increments. ##### Concurrent Spanning Tree Algorithm - `example/span.v`. This module presents the re-implementation of a concurrent and recursive spanning-tree algorithm already verified in FCSL. The Span resource keeps a pointer-based graph in its joint heap ghost `σ_j` , and the self and other ghosts, `μs` and `μo`, are finite sets of marked pointers which keep track of the nodes in the graph that have already been visited by span. We use morphisms for projecting the components of the state, as well as to implement a quiescent client, which hides the helping sub-threads of the spanning-tree graph from the parent thread’s environment.