4661956
doi
10.5281/zenodo.4661956
oai:zenodo.org:4661956
user-icst2021
A Practical Method for API Testing in the Context of Continuous Delivery and Behavior Driven Development
Bennett, Brian Elgaard
info:eu-repo/semantics/openAccess
Creative Commons Attribution 4.0 International
https://creativecommons.org/licenses/by/4.0/legalcode
Combinatorial Testing, API Testing, Behavior Driven Development, Domain Driven Design, Continuous Delivery, Industry, Application
<p>Abstract—Enterprises are increasingly adopting an API-first approach to connect and expose software services. Saxo Bank is no exception to this. </p>
<p>Crafting test suites for such APIs can seem straight forward due to the headless nature, but our experience shows that test suites often have two problems. The first problem is that execution of tests tends to fail and pass in seemingly nondeterministic ways (tests are flaky). The second problem is that functional coverage is not clearly documented.</p>
<p>We have found that both problems stem from a lack of explicit focus on initial context (IC), a concept from behavior driven development. When a test is flaky it is often because actual IC in the test environment is not as required by the test. When functional coverage is not clear, it is most often because a systematic analysis involving IC was not performed.</p>
<p>We propose a method for test analysis in which we include IC in the input space when analyzing functional coverage for an API, thereby including anything which can influence the outcome of test cases.</p>
<p>Establishing IC is in general a hard problem. We have found that focus on the bounded context, a concept from domain driven design, of the system under test is a practical way to establish relevant IC.</p>
<p>Experience with Saxo Bank's Open API shows that this method allows testers and developers to cooperate continuously, producing test plan documents which include the reasoning behind functional coverage. Explicit focus on IC in automated test case implementations turns flaky tests into tests which report on required IC in a test environment. The method easily generalizes to all levels of API tests. </p>
Zenodo
2021-04-04
info:eu-repo/semantics/lecture
4661955
user-icst2021
1619081142.943306
797153572
md5:8d8195a453b68f56a739b5f42c2a67d8
https://zenodo.org/records/4661956/files/IWCT 20213.mp4
public
10.5281/zenodo.4661955
isVersionOf
doi