Improving formal verification with portfolio-based runtime techniques

Formal verification is an approach of using mathematically precise representations and algorithms to check properties of a given program or model. Formal verification is gaining increasing importance, especially in safety-critical domains. However, formal methods are computationally complex, which has resulted in various efficient algorithms tailored for different application domains.

In most cases, choosing the right algorithm and configuration for a given problem requires expert knowledge (e.g. which abstraction method to use during the verification). Even an expert might need to execute several configurations before finding one that performs well on the given verification task. But time and resources are limited in most cases.

My goal in this work is to propose techniques that help utilize the available time more efficiently. Assuming a configurable verifier tool, an input task, a time constraint and sequential execution, these methods select and dynamically change verification configurations forming a complex portfolio. The novelty of the method is the usage of not just information from the input task, but also runtime progress information to intervene in the current execution or to select when and which configuration to switch to.

To show how these techniques can be tailored to a tool in practice, I realized these in the tool Theta, focusing on C program verification. Improvements include adding a runtime enhancement to the CEGAR loop, which is capable of detecting when the algorithm is stuck. A portfolio offering a diverse set of configurations with algorithm selection is also added, such that it complements the runtime approach to increase the chance of success.

For evaluation a subset of the benchmarking tasks of the International Competition on Software Verification (SV-COMP) are used. SV-COMP is widely regarded as a de-facto standard benchmark set of C program verification tasks. The recommended improvements were compared to several generally well-performing configurations, executed one at a time and in a naive sequential portfolio as well. The tool with the added improvements proved capable of solving more tasks significantly faster than those in the baseline configurations.

To summarize, I designed approaches for efficient verification by automating some of the configuration and algorithm selection tasks requiring expert knowledge. The approaches are general and applicable in any verification framework. To evaluate them, I realized these techniques in a specific verifier framework, showing how it can improve the tool's performance.