File uploads: We have fixed an issue which caused file uploads to fail. We apologise for the inconvenience it may have caused.

Published August 16, 2022 | Version v2
Dataset Open

SAT-Inspired Higher-Order Eliminations

  • 1. Vrije Universiteit Amsterdam

Description

This is the package containing the raw evaluation data for the paper "SAT-Inspired Higher-Order Eliminations" by Jasmin Blanchette and Petar Vukmirović.

The problems used for the evaluation are located in the "problems" directory. The seven categories are

    seventeen_th0 (called S0 in Fig. 1 of the paper)
    seventeen_th1 (called S1 in Fig. 1)
    tptp_th0 (called TH0 in Fig. 1)
    tptp_th1 (called TH1 in Fig. 1)
    tptp_cnffof (called CF in Fig. 1)
    tptp_tf0 (called TF0 in Fig. 1)
    tptp_tf1 (called TF1 in Fig. 1)

The empirical results are located in the "results" directory, under the following names, corresponding to the category names above:

    seventeen_th0_results.csv
    seventeen_th1_results.csv
    tptp_th0_results.csv
    tptp_th1_results.csv
    tptp_cnffof_results.csv
    tptp_tf0_results.csv
    tptp_tf1_results.csv

The CSV files were produced by StarExec. Each nonheader row gives the prover's performance on one problem. For example, the row

    74437543,Problems/AGT/AGT036^1.p,2900058,Zipperposition---2.2pre-hoelim-v2,2410,hlbe-in,92437,complete,2.09374,0.983524,1684480.0,Theorem,Theorem,THM-Ref,Ref,THM

in "tptp_th0_results.csv" indicates that the HLBE inprocessing mode of Zipperposition ("hlbe-in") was able to prove the TPTP problem "AGT036^1.p", as indicated by the "THM" result in the last column. "THM" and "UNS" (unsatisfiable) correspond to a successful proof; other outcomes are considered failures.

Figure 1 was generated using the script "script/gen_figure.py", which must be run from within the "script" directory.

The "binaries" directory contains the StarExec package used to run the evaluation. The package is called "bin" in accordance with StarExec conventions. Inside it, "zipperposition" and "eprover-ho" are the 64-bit Linux binaries for the Zipperposition prover and its E backend, and the other files are scripts used to run various configurations in time slices. When running the scripts locally, set the environment variables "STAREXEC_CPU_LIMIT" and "STAREXEC_WALLCLOCK_LIMIT" to suitable time limits in seconds.

Zipperposition was compiled from the repositiory version with the git commit hash 2a66166453ac32c0 on the "wip_ho_elimination_techniques" branch. E was compiled with the "--enable-ho" configuration option from an unspecified repository version. The Zipperposition and E repositories are available online (https://github.com/sneeuwballen/zipperposition and https://github.com/eprover/eprover).

Files

satelimhol_data.zip

Files (484.4 MB)

Name Size Download all
md5:407e706e12860e49683e5e1334ef3261
484.4 MB Preview Download

Additional details

Funding

Matryoshka – Fast Interactive Verification through Strong Higher-Order Automation 713999
European Commission