Artifact for Type Tailoring

DOI: https://doi.org/10.5281/zenodo.10578596

Overview

The purpose of this artifact is to provide running code to support the claims in section 2 and subsection 3.8 of the paper. The programs demonstrate type tailoring in a variety of languages and settings.

This artifact does not support other sections:

  • Section 3 (excepting subsection 3.8) is a qualitative analysis of languages. Citations in the paper support its claims.
  • Section 4 is a design space analysis. It builds on section 3.
  • Section 5 presents reasoning principles.

All the code is under the claims/ directory. We call this directory "claims root" below.

This artifact is available as source code or as a Docker container.

We recommend the Docker container, as it includes all dependencies, but because we built the Docker on an x86 machine it may not run on Apple Silicon. When we tested on an Apple M1 Pro, the Julia section failed. Everything else was okay, but ran relatively slowly.

Step-by-Step Instructions

This is a small artifact. Running the tests in the Docker container should take about an hour on a typical x86 machine.

Make sure to enable unlimited scrollback on your terminal so that you can refer to the output of the commands below.

Prerequisites (not for Docker, source builds only)

Note: If you use the Docker container, skip ahead to the next section Running the artifact scripts.

Install the following languages:

Racket Packages

Install the following Racket packages:

raco pkg install --batch --auto --no-docs threading levenshtein pfds syntax-sloc type-expander

(If you did not listen earlier and installed Minimal Racket, then install redex and racket-test too.)

Enter the claims/ directory ("claims root").

Clone the following packages; we need specific versions:

# Rhombus
git clone https://github.com/racket/rhombus-prototype/
cd rhombus-prototype
git checkout 6b3f81e6ffd9ea0cc504be7209bbe9bf9d48b14e
raco pkg install --batch --no-docs --auto

cd .. # (should be back to claims root)

# Trivial
git clone https://github.com/bennn/trivial.git
cd trivial
git checkout 6da6a2edea1912c7b28d0bcaca5ffe09bf247ca5
raco pkg install --batch --auto --no-docs ./trivial

cd .. # (should be back to claims root)

# Dyn
git clone https://github.com/ashton314/rhombus_dyn.git
cd rhombus_dyn
git checkout 7eb9fc304b2ebe2d82964328766681fc7de93c29
raco pkg install --batch --auto --no-docs

cd .. # (should be back to claims root)

# Shplait
git clone https://github.com/mflatt/shplait.git
cd shplait
git checkout e8da0e1a8a22e5d1f0b1ec6603697471a571b780
raco pkg install --no-docs

cd .. # (should be back to claims root)

Estimated time to run: 5–10 minutes

Julia Packages

Run the following commands to install packages listed in the Julia manifest file:

cd julia/ # (subdirectory of claims root)
julia --project=.

You should now be in the Julia REPL and see a prompt that says julia>.

Enter the package manager mode of the REPL by pressing ] (close square bracket):

julia>]

The prompt should change to (julia) pkg>.

Run one command in the package manger REPL:

(julia) pkg> instantiate

This will install the packages needed.

Estimated time to run: 2–3 minutes. Julia precompiles all its packages, and this may be slow at times.

Exit the Julia REPL by hitting Ctrl-D.

Elixir Packages

Run these commands:

cd elixir/ # (subdirectory of claims root)
mix local.hex
mix archive.install hex phx_new
yes n | mix phx.new demo_phoenix

Running the artifact scripts

The remaining tests should take 20–30 minutes to run in their entirely.

Starting the Docker container

You have two options for getting the Docker container. First, you can download type-tailoring-container.tar.gz from Zenodo, decompress the archive, load it into Docker, and start the container:

# option 1 of 2
gunzip type-tailoring-container.tar.gz
docker load -i type-tailoring-container.tar
docker run --rm -it researcherw/type-tailoring:latest

Alternatively, you can simply pull the container from DockerHub and start the container:

# option 2 of 2
docker run --rm -it researcherw/type-tailoring:latest

That should open a container with all of the languages and dependencies installed. The Docker comes with a push_button script that runs all of the scripts in sequence. Additionally, there are commands to run smaller portions of the artifact one at a time. The commands should work regardless of what directory you are in.

Typed Racket code listings from §1 and §2.1

Docker fast-track: run tr_examples to execute all Typed Racket examples at once. Or, follow the steps below for source builds.

From the claims root, run these commands:

cd tr_examples
for f in sec{1,2}*.rkt; do echo "Testing $f"; raco make -v $f >/dev/null && raco test $f; echo; done

That will run the five .rkt files in the examples directory.

<details><summary>Output (click to view)</summary>

Testing sec1-listing.rkt
raco test: "sec1-listing.rkt"
59

Testing sec2-listing1.rkt
raco test: "sec2-listing1.rkt"
hello world
6 tests passed

Testing sec2-listing2.rkt
raco test: "sec2-listing2.rkt"
6 tests passed

Testing sec2-listing3.rkt
raco test: "sec2-listing3.rkt"
string properties:
 #hasheq((A . A-⊥35648) (B . #f) (C . C-⊥58215) (Conn . #f) (DB . #f) (F . ()) (I . I-⊥49248) (L . L-⊥53551) (R . ("SELECT breed FROM Cats")) (S . "(SELECT breed FROM Cats)") (V . V-⊥53853))
3 tests passed

Testing sec2-listing4.rkt
raco test: "sec2-listing4.rkt"
2 tests passed

</details>

Estimated time to run: <1 minute.

Typed Racket code listings from §3.8

From the claims root, run these commands:

cd tr_examples
racket sec3-listing.rkt

<details><summary>Output (click to view)</summary>

"sword"

</details>

Next, verify that vector-ref was elaborated to unsafe-vector-ref; run this command:

raco expand sec3-listing.rkt | grep --color 'unsafe-vector-ref\|$'

You should see unsafe-vector-ref highlighted in the output.

Estimated time to run: 1 minute.

Typing Regular Expressions

Docker users: either follow these instructions or run tr_benchmarks.

From the claims root, run these commands:

cd tr_benchmarks/regexp/hits
rm -f DATA.rktd
racket README.md

Estimated time to run: 5–10 minutes.

When that finishes, there will be a file called DATA.rktd. Run this command to count hits and misses:

perl hit_counter.pl DATA.rktd

Estimated time to run: 1 second.

That should print out:

Hit: 262 Miss: 3

Predicting Vector Bounds

cd tr_benchmarks/vector
rm -f DATA.rktd
racket README.md

Estimated time to run: 5–10 minutes.

When that finishes, there will be a file called DATA.rktd. Run this command to count hits/misses:

perl hit_counter.pl DATA.rktd

Estimated time to run: 1 second.

This should print out:

Hit: 106 Miss: 499

Concise GUI Subtypes (7GUI)

From the claims root, run:

cd tr_elaborating_types/7gui
grep --color -A 3 -n 'define-type-canvas' task-6.rkt

Estimated time to run: 1 second.

The grep command will show a type declaration with 2 members (instead of 13 normally required) from Figure 1 in the paper.

Functional-Style Object Types (Zombie)

From the claims root, run:

cd tr_elaborating_types
# Use of Zombie in a type
grep --color -A 19 -n '(: new-zombie (-> Posn Zombie))' zombie.rkt
# Zombie type definition
grep --color -A 4 -n 'make-fake-object-type\* Zombie$' zombie.rkt

Estimated time to run: 1 second.

The first grep command shows a function that returns a value of type Zombie.

The second grep command shows how the Zombie type is defined, corresponding to the right half of the Without/With Tailoring code.

Type Expanders

Docker fast-track: this ran as part of tr_examples.

From the claims root, run:

cd tr_elaborating_types
racket homogeneous-list.rkt

Estimated time to run: 2 seconds

You should see the following output:

'("hello" "a" "b" "c" "d")
'("moon" "a" "b" "c" "d")

Using Dyn in Shplait

Docker fast-track: run shplait_claims to execute all Shplait/Rhombus examples.

From the claims root, run the following commands:

cd rhombus
racket section2-listing2.rhm
racket let_cc.rhm

Estimated time to run: <1 minute

racket section2-listing2.rhm should output:

#true

Running let_cc.rhm should not produce output.

Run these commands to patch Shplait with the Dyn annotation:

cd ..        # back to claims base
cd shplait   # where we cloned the library
git apply ../rhombus/dyn_annotation.patch
raco make -v -l shplait

Estimated run time: 2–3 minutes.

Now, with Dyn swapped out for type annotations in the Shplait source, and Shplait rebuilt, rerun the interpreter:

cd ..        # back to claims base
cd rhombus
racket let_cc.rhm

Estimated time to run: <1 minute, approximately same as last time

Expect no output; the interpreter should run without errors.

Static Arrays in Julia

Docker fast-track: run julia_claims to execute all Julia examples.

From the claims root directory, run the following commands:

cd julia
julia --project=.

That will start a Julia REPL. Startup should take less than a minute, unless it reports needing to precompile packages, in which case it may take up to 5 minutes.

Inside the Julia REPL, run the following commands:

julia> include("main.jl")
julia> run_normals()
julia> run_statics()

Estimated time to run: 5 minutes for run_normals(), 1 minute for run_statics()

This is a scaled-down version of the experiment reported in the paper. Refer to the comments in main.jl for instructions on running the full experiment (estimated time: 4 hours).

At the end of each run, you should see benchmark results like the following:

julia> run_normals()
Without StaticArrays
BenchmarkTools.Trial: 10 samples with 1 evaluation.
 Range (min … max):  258.541 μs … 318.708 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     271.000 μs               ┊ GC (median):    0.00%
 Time  (mean ± σ):   277.000 μs ±  18.438 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

  ▁  ▁  █     █      ▁            █                           ▁  
  █▁▁█▁▁█▁▁▁▁▁█▁▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█ ▁
  259 μs           Histogram: frequency by time          319 μs <

 Memory estimate: 421.88 KiB, allocs estimate: 3000.

julia> run_statics()
With StaticArrays
BenchmarkTools.Trial: 10 samples with 1 evaluation.
 Range (min … max):  37.708 μs … 38.041 μs  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     37.792 μs              ┊ GC (median):    0.00%
 Time  (mean ± σ):   37.804 μs ± 94.133 ns  ┊ GC (mean ± σ):  0.00% ± 0.00%

         ▄      █                                              
  ▆▁▁▁▁▁▁█▁▁▁▁▁▁█▁▁▁▁▁▁▁▁▁▁▁▁▁▁▆▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▆ ▁
  37.7 μs         Histogram: frequency by time          38 μs <

 Memory estimate: 0 bytes, allocs estimate: 0.

Leave the REPL by typing Ctrl-D.

Still in the julia directory, run this command:

julia --project=. minor_examples.jl

Estimated time to run: ~5 seconds

You should see approximately this output:

[1]
[0.0, 0.0, 0.0]
[1, 4, 9, 16, 25, 36, 49, 64, 81]
[0.3827773322541715 0.09819782318335457 0.2915972870916304; -0.7615538511444314 0.6184631261009615 0.8003752355383472; 1.0744030057167748 0.6839913385878595 0.08534538553782166]
SVector{3, Float64}

The last line is a randomly generated matrix.

Verified Routes by Default

Docker fast-track: run elixir_claims to execute all Elixir examples.

Return to the claims root directory. Run the following commands:

cd elixir
cd demo_phoenix
grep --color -n 'verified_routes\|$' lib/demo_phoenix_web.ex
grep --color -n '~p"[^"]*"\|$' lib/demo_phoenix_web/components/layouts/root.html.heex

Estimated time to run: ~5 seconds

The end!