Artifact: Deep Static Modeling of invokedynamic

This artifact contains the evaluation benchmarks for the paper "Deep Static Modeling of invokedynamic" by G. Fourtounis and Y. Smaragdakis (ECOOP '19).

System requirements

For the paper numbers, the benchmarks were analyzed on a machine with an Intel Xeon CPU E5-2667 v2 3.30GHz with 256 GB of RAM, running Linux (Fedora 29).

As a base system, the benchmarks have also been analyzed on a machine with an Intel Core2 Duo CPU (T6570, 2.10GHz) with 8GB RAM, running Linux (Ubuntu 16.04).

This artifact bundles a VirtualBox "appliance" (tested with VirtualBox 5.1.38). The hardware capabilities observed by the virtual machine should fulfill the above requirements. The virtual machine contains a "user" account (password: "user") with sudo capabilities, so that additional software may be installed.

Installation and Use

  • Import "invokedynamic-benchmarks.ova" in VirtualBox via "File" -> "Import Appliance". Start the virtual machine.

  • When the virtual machine starts and the desktop appears, open a terminal (menu -> "System Tools" -> "LXTerminal").

  • Change to the benchmarks directory:

  cd ${HOME}/invokedynamic-benchmarks

Analyzing the Microbenchmark Suite

To analyze the microbenchmark suite (Section 5.1 in the paper), run:

  ./analyze-microbenchmarks.sh

The programs should be analyzed without errors and a table with times should be printed (same as Figure 8 in the paper). A list of successfull tests should also appear ("Running check: ...").

The analysis of these programs covers the set of features mentioned in Section 5.1 of the paper. The output also mentions the features checked successfully ("Feature: ..."). For further manual inspection:

  • The sources of the programs are under $DOOP_HOME/tests.

  • The test scenarios for the features are found in the following locations:

    $DOOP_HOME/src/test/groovy/org/clyze/doop/TestLambdasMethodReferences.groovy

    $DOOP_HOME/src/test/groovy/org/clyze/doop/TestInvokedynamic.groovy

    $DOOP_HOME/souffle-logic/addons/testing/TestInvokedynamic.dl

  • The Jimple intermediate representation can be found in $DOOP_HOME/out/context-insensitive/test-X/facts/jimple, where X is one of "104-method-references", "107-lambdas", and "115-invokedynamic". This is only available when --generate-jimple is given in the tests (instead of --Xlow-mem).

Analyzing the Dynamic Test Suite

To analyze the test suite of Sui et al. (Section 5.2 in the paper), run:

  ./bench-invokedynamic.sh

The script generates the table of Figure 9 in the paper.

For further manual inspection:

  • The sources of the benchmarks can be found in directory ${HOME}/invokedynamic-benchmarks/dynamic-benchmark.

  • The Jimple intermediate representation can be found in $DOOP_HOME/out/context-insensitive/X/facts/jimple, where X is one of "lambda-consumer", "lambda-function", "lambda-supplier", and "dynamo-reflection". This is only available when --generate-jimple is given in bench-invokedynamic.sh (instead of --Xlow-mem).

Troubleshooting

  • The virtual machine may run out of disk space if benchmarks are run manually and all analysis results are kept. In that case, delete doop/out or doop/cache to make space and rerun the analysis.

  • If the microbenchmark suite fails with a Gradle lock error, there may be a stale Gradle process still running. Stop it (kill -9 <PID>) and rerun the analysis.