Statically Resolvable Ambiguity

System Requirements

The docker image is large (10.4 GB), as is the data output by a complete run of the benchmarks (an additional 3.1 GB), thus we recommend a computer with a decent amount of free disk space.

Note also that we have only tested the image on Linux and Mac (ARM), and that Mac runs docker images through a VM, thus its runs will be significantly slower.

The Artifact

This artifact contains everything related to the Statically Resolvable Ambiguity paper:

• The mechanized proof of statically resolvable ambiguity.
• The modified versions of the OCaml compiler.
• The library implementing our grouper.
• The benchmarking script.
• The benchmarking data used in the paper.
• The Jupyter notebook analyzing the data and producing the plots.

These are available in two forms:

• As a docker image, with all of the above and all relevant executables and dependencies built and installed.
• As a compressed archive, only containing the source code for all of the above.

The latter is smaller and useful to just look at the code without downloading a very large docker image, but requires setting up an appropriate environment if you wish to run anything. We thus strongly recommend using the docker image if you wish to run any of the experiments or try out our modified OCaml compiler.

Kick the Tires - Basic Instructions for Setup

First, make sure you have Docker installed and configured (see docs.docker.com/get-docker/ for instructions).

Next, download the docker image and run the container:

# Note that depending on how your docker installation is configured
# you may need to add 'sudo' before all 'docker' commands.

# This first command may take a few minutes, the image is large
docker load --input static-resolvable.tar.gz

# The second command should be quick
docker run -p 8888:8888 -it --name static-resolvable-container static-resolvable

At this point you should have a shell open in the folder /root in the container. You can exit either using Ctrl+D or exit.

When you later want to resume the container you can use:

docker start -ia static-resolvable-container

The container starts with the shell in directory /root, which contains all relevant source code, previously produced benchmark data, and a Makefile to assist in running the various components. We suggest these commands be run during the "kick the tires" phase:

make coq            # Compile the coq proof, found under /root/coq,
                    # takes approx. 7min.

make coq-check      # Check that there are no 'admit's in the proofs

make ast-check      # Check that 'unamb' parses the same as 'original',
                    # takes approx. 2 minutes. Each equal parse prints
                    # a '.', each non-equal parse prints a '!'.

make bench-small    # Run a small subset of the benchmarks, takes approx. 9min.
                    # Output is placed in /root/data/data-log-small.csv, which
                    # will be read by a Jupyter notebook later. Progress will
                    # be printed in the form of a '.' for a successful parse
                    # and 'F' for a failed parse. We expect a number of parse
                    # failures, see reasoning in the Jupyter notebook later,
                    # under the heading 'Parse failures, totals'.

make jupyter        # Open the Jupyter notebook to generate and examine plots.
                    # Copy the link printed starting with '127.0.0.1' and open
                    # it in your web browser; the 'docker run' command above
                    # exposes the port outside the container. If the notebook
                    # does not open immediately, double-click 'Analysis.ipynb'
                    # in the file tree to the left.

The Jupyter notebook contains further instructions on how to run it and generate the plots. To close the notebook either hit Ctrl+C in the terminal followed by y, or select File > Shut Down in the notebook in the browser.

More Information

The docker image and compressed archive both contains a README as markdown (README.md) and as a rendered PDF (README.pdf). These contain all of the instructions above, as well as more information on how to verify our various claims and run variations of our benchmarks and experiments.

We thus suggest that more extensive exploration should begin by examining one of those documents.