# Artifact Evaluation Instructions The virtual machine can be run by execute the `run.sh` script. The account name is `artifact` and the password is `password`. ## Kick-the-tires Run `make kick-the-tires`. You will see the program running and get back the results similar to the following: ``` "app(app(Data.Maybe.fromMaybe, x), app(Data.Maybe.listToMaybe, app(Data.Maybe.catMaybes, xs)))" "Time: 2.008083928s" Success 2.008083928 ``` This indicates that you have set up the environment successfully and may proceed to the next section. ## Full experiment Run the command `make complete` to collect data for Fig 15-18 of the original submission. It takes about 12-16 hours to complete, so we recommend to left it run over night. Note that the VM sometimes kills some processes running out of memory and it is not an error. After running the experiments, you may run `make plots` to generate the 4 figures in the paper. The images are generated inside the VM, you may run `scp -P 5555 artifact@localhost:~/fig\*.pdf` to download and examine them on your local machine. ## Build from source code To build this project from source code, you need to install the Haskell tool [stack](https://docs.haskellstack.org/en/stable/install_and_upgrade/). Once you have installed stack, you may run `cd compact-coupled-terms; stack build` to build the project. Note: to run the experiments locally, you will also need to follow the instructions to install hoogle+, which requires to install [z3](https://github.com/Z3Prover/z3) locally. But ~if you only want run ECTA or Hectare locally~, it is unnecessary to install that. ### Run Hectare The following is an example command to run a synthesis benchmark: ```bash stack exec -- compact-coupled-terms-exe 'Benchmark {bmName="firstJust", bmSize=5, bmSolution=(Term "app" [Term "app" [Term "Data.Maybe.fromMaybe" [], Term "x" []], Term "app" [Term "Data.Maybe.listToMaybe" [], Term "app" [Term "Data.Maybe.catMaybes" [], Term "xs" []]]]), bmArguments=[("x",TVar "a"),("xs",TCons "List" [TCons "Maybe" [TVar "a"]])], bmGoalType= (TVar "a")}' ``` A benchmark includes five parts: - benchmark name - the size of desired solution size - the desired solution - the argument types of your query - the return type of your query If you don't know the desired program for your type query, you may provide a slightly large size limit and desired solution of fake terms as follows: ```bash stack exec -- compact-coupled-terms-exe 'Benchmark {bmName="firstJust", bmSize=5, bmSolution=(Term "dummy" []), bmArguments=[("x",TVar "a"),("xs",TCons "List" [TCons "Maybe" [TVar "a"]])], bmGoalType= (TVar "a")}' ``` Hectare will produce all programs of size up to the provided size limit that satisfy the given type query. To save the time, we did not decode the generated programs as that when you provide the desired solution, but it is easy to be added later. **Interpretation of `Term`s** When running Hectare, expressions of type `Term` are used in both inputs and outputs. To help you write benchmarks and read the synthesis results more easily, we provide some descriptions on the syntax of `Term`s. Since Hectare only supports application terms, so in this artifact, a `Term` is - either a symbol with no arguments, such as `Term "x" []` for a variable `x`, `Term "listToMaybe" []` for a function `listToMaybe`, - or an application `f x`, written as `Term "app" [Term "f" [], Term "x" []]`. Note that our `app` terms always take exactly two inputs. Applications like `f x y` will be represented as two nested `app`s. In this style, the term for `fromMaybe x (listToMaybe xs)` is written into three apps. Let us first add parenthesis to different applications: `((fromMaybe x) (listToMaybe xs))` and then we translate this into our `Term` data structure as `Term "app" [Term "app" [Term "fromMaybe" [], Term "x" []], Term "app" [Term "listToMaybe" [], Term "xs" []]]` **Interpretation of `Type`s** In this artifact, Hectare supports four kinds of features in types: type variables, data types, function types and type classes. They are represented as follows: - type variables: `a` is written as `TVar "a"`; - data types: `DT args` is written as `TCons "DT" [representation of args]`; - function types: function types are usually seen in higher-order arguments, and a function type `a -> b` is written as `TFun (TVar "a") (TVar "b")`; - type classes: we adopted the same style from Hoogle+, where, type classes are represented as a special data type with a predefined prefix. For example `Eq a` is written as `TCons "@@hplusTC@@Eq" [TVar "a"]`. With the same scheme, we may write the arguments of the query `Eq a => [a] -> [a]` as `TCons "@@hplusTC@@Eq" [TVar "a"]` and `TCons "List" [TVar "a']`. ### Build Other Projects on ECTA The ECTA library is available in `~/compact-coupled-terms/src/Data/ECTA`. The supported operations are separated in three files: - `src/Data/ECTA/Internal/ECTA/Operations.hs`: operations on ECTA nodes and edges, including union, intersection, reduction, etc. - `src/Data/ECTA/Internal/ECTA/Enumeration.hs`: fast enumeration algorithm for ECTAs, including the implementation of choose and suspend. - `src/Data/ECTA/Internal/Paths.hs`: operations on ECTA constraints, including consistency check, constraint creation and union, etc. Here are the code indices for the important algorithms about ECTA: - intersection: the method `intersect` of the file `src/Data/ECTA/Internal/ECTA/Operations.hs`; - static reduction: the method `reducePartially` of the file `src/Data/ECTA/Internal/ECTA/ECTA/Operations.h`. This method takes an ECTA node and reduces each edge inside this node with the method `reduceEdgeIntersection`. Inside `reduceEdgeIntersection`, an important step is to check whether the equality constraints are satisfiable. This check is delegated to the method `reduceEqConstraints`. If the check fails, the algorithm will exclude the failed edge. - dynamic reduction: the entry point of this algorithm is the method `getAllTerms` in the file `src/Data/ECTA/Internal/ECTA/Enumeration.hs`. In detail, the `choose` and `suspend` steps correspond to the two case branches of the method `enumerateNode`. You may refer to the interfaces in these files to build your own projects over ECTAs. We recommand that it is easier for you to write down the encoding on paper first and verify that ECTA can be used to encode your problem. Once you know how to encode the problem, you may then write the code to test your idea.