Reducing DNN Properties to Enable Falsification with Adversarial Attacks

This artifact accompanies the paper Reducing DNN Properties to Enable Falsification with Adversarial Attacks.
In this artifact, we provide the benchmarks and scripts for reproducing the results of our study,
and we also provide our tool, DNNF for running falsification methods such as adversarial attacks on DNN property specifications specified using the DNNP language of DNNV.

While many DNN verification techniques have been introduced in the past few years to enable the checking of DNN safety properties, these techniques are often limited in their applicability, due to simplifying assumptions about DNN structure or to high computational cost.
Falsification is a complementary approach to verification that seeks only to find violations to a safety property.
In the context of DNNs, adversarial attacks can be viewed as falsifiers for DNN local robustness properties.
While these techniques often scale to large real-world DNNs, they are currently limited in the range of properties they can falsify.

In Reducing DNN Properties to Enable Falsification with Adversarial Attacks, we introduce an approach for reducing a DNN and an associated safety property -- a correctness problem -- into an equivalid set of correctness problems formulated with robustness properties which can be processed by existing adversarial attack techniques.
We implement the approach in a tool which we call DNNF, and we perform a study demonstrating that property reduction yields a cost-effective approach to find violations of DNN correctness problems.