Published August 27, 2021 | Version v1
Dataset Open

The 2020 Comparison of Tools for the Analysis of Quantitative Formal Models: Results and Reproduction

  • 1. University of Twente
  • 2. Saarland University

Description

This archive contains detailed results from QComp 2020 as well as the necessary scripts and data to reproduce them.

Visit http://qcomp.org for more information for QComp.

Overview of Contents

- `qcomp.org/` contains the state of our website from the timepoint of the competition. This includes:
  - All benchmark files, browsable at `qcomp.org/benchmarks/index.html`
  - Detailed competition results in a human-readable format, browsable at `https://qcomp.org/competition/2020/`
- `logs/` contains the raw logfiles and data gathered by our scripts
- `scripts/` contains scripts to replicate the whole competition
- `toolpackages/` contains a package for each participating tool which includes
  - Instructions for obtaining and installing the tool
  - a file `invocations.json` listing the commandlines used in QComp 2020
  - a file `tool.py` providing functionalities to obtain the result from the tool output.

Files

QComp2020Replication.zip

Files (738.1 MB)

Name Size Download all
md5:263b15ade511a8688c1f69438d7f823b
738.1 MB Preview Download

Additional details

Related works

Cites
Dataset: 10.5281/zenodo.2628302 (DOI)
Conference paper: 10.1007/978-3-030-17502-3_5 (DOI)
Is cited by
Conference paper: 10.1007/978-3-030-83723-5_15 (DOI)

Funding

European Commission
POWVER - Power to the People. Verified. 695614
European Commission
FUN2MODEL - From FUnction-based TO MOdel-based automated probabilistic reasoning for DEep Learning 834115
European Commission
FRAPPANT - Formal Reasoning About Probabilistic Programs: Breaking New Ground for Automation 787914

References

  • On Correctness, Precision, and Performance in Quantitative Verification - QComp 2020 Competition Report, ISoLA 2020