Readme

Building

• Install docker on your machine
• Run docker build -t ssp . in the directory where the code is extracted. If you have Gurobi 9.1 and Java version >= 17 installed, you can also run the code directly on your machine by replicating the steps from the Dockerfile.
• Obtain a WLS licence from Gurobi (see https://license.gurobi.com/, free for academic use) and save the gurobi.lic somewhere

Running

• To run a command, execute docker run -it -v <path to gurobi.lic>:/opt/gurobi/gurobi.lic ssp <command>

• Optional: When running experiments, also add -v <some path>:/app/output to the docker call to obtain the output files on your machine. So, the complete command is

  docker run -it -v <path to gurobi.lic>:/opt/gurobi/gurobi.lic -v <some path>:/app/output ssp <command>

• The models can be found in the folder /app/models

• Running our implementation always has the form

  bin/ssp-risk <model transitions> <model labels> <goal label> reach <methods: vi,lp> <threshold> output/<output name>.json

• All relevant data can be found in <some path>/<name>.json

• To specify memory limits, ensure that docker has enough memory available and add -e JAVA_OPTS="-Xmx10G" directly after docker run.

Evaluation

Run the following commands for the respective models:

• Grid4: bin/ssp-cvar models/gridworld/gridworld.04.d.tra models/gridworld/gridworld.04.d.lab station reach lp,vi 0.10 output/gridworld.04.json
• Grid8: bin/ssp-cvar models/gridworld/gridworld.08.d.tra models/gridworld/gridworld.08.d.lab station reach lp,vi 0.10 output/gridworld.08.json
• Grid16: bin/ssp-cvar models/gridworld/gridworld.16.d.tra models/gridworld/gridworld.16.d.lab station reach lp,vi 0.10 output/gridworld.16.json
• Grid32: bin/ssp-cvar models/gridworld/gridworld.32.d.tra models/gridworld/gridworld.32.d.lab station reach vi 0.10 output/gridworld.32.json
• FireWire: bin/ssp-cvar models/firewire/firewire.tra models/firewire/firewire.lab absorbing reach vi 0.10 output/firewire.json
• WLAN: bin/ssp-cvar models/wlan/wlan2.tra models/wlan/wlan2.lab absorbing reach vi,lp 0.10 output/wlan.json
• Walk: bin/ssp-cvar models/random_walk/random_walk.tra models/random_walk/random_walk.lab goal reach vi "range:0.001,0.1,100|logrange:-7,-3,100" output/walk-multi.json

We have excluded the variants which time out or run out of memory, to include them simply change vi / lp to vi,lp in the respective commands. To view the results, simply open the created JSON files.

Run python3 eval.py walk-multi.json to output a CSV for the different threshold performances.

By increasing the logging level of Java’s default logging facility, further details of the computation can be viewed (note that the statistics computation may increase runtime by a few percent).

Modifying

• The sources.tar.gz contain all sources of our implementation (both Java and a python prototype). In fact, the binary executed in the evaluation is compiled from the contained sources. If you want to modify the sources, simply re-package the sources.tar.gz and re-build the docker image.

• The models are written in the PRISM modelling language. They can be modified and then exported using PRISM (a release can be obtained at https://www.prismmodelchecker.org/), using the call

  prism <model>.nm -exportmodel .all -const <constants>

  This produces (among others) the required .tra and .lab files. The constants we used to create the models can be found in the .const files in each model folder.