Specification-based Test case generation for API misuse detection tools

Misuses of application interfaces (APIs) can cause serious programming mistakes. Past research has proposed to identify and thus avoid misuses through static and dynamic API misuse detection tools. But these tools themselves can be faulty, causing distracting false warnings or otherwise overlooking actual misuses. In this work, we thus present a novel approach that facilitates test automation for API misuse detectors by automatically generating test cases from existing API usage specifications. Our specific prototype APItestGen supports test generation from the API specification language CrySL. Its realization required us to overcome interesting challenges because CrySL specifications not only constrain method parameters but also method-call sequences and combinations of both on multiple related API objects. We evaluated APItestGen by generating tests for the Java’s entire default crypto API, the JCA, and using the tests to validate the static analysis implemented in the CogniCryptSAST static API misuse checker. The evaluation revealed previously unknown bugs in the analyzer.