========================================== Artifact Evaluation for Paper id 322 ========================================== Note: This README file is Version 2.0. We have made some changes in "ISSTA21.ova" VM image. The changes are explained in "/home/issta21-322/README" file. --------------------------------------------------------------------------------------------------------------------------- A. This is the README file to navigate the artifacts. B. This artifact contains "Main_Artifact-ISSTA-21.zip" and "ISSTA21.ova" files. C. "Main_Artifact-ISSTA-21.zip" has two subfolders "CBMC" and "CUSTOM". Also it contains "README" file which explains all the files and folders and "ISSTA-2021-Artifact-Main-Experiment.pdf" workbook that shows all the results. D. "ISSTA21.ova" is a VM image. Kindly navigate to home folder "/home/issta21-322/" to get "README" file. We also provide the essential requirements below: 1. Oracle VM VirtualBox 6.1 manager (https://www.virtualbox.org/wiki/Downloads) 2. Linux Ubuntu 16.04, Processor: Intel® Core™ i7-9700 CPU @ 3.00GHz, Graphics: llvmpipe (LLVM 6.0, 256 bits), OS Type 64 bit, RAM 30.7 GB, Disk 30.6 GB. Note: We have also checked our artifact with 12 GB RAM, it works with some time difference. So, reviewers make a note of the RAM size and consider the results. 3. Username: ISSTA21-322 , Password: issta21-322 E. We claim for all the three badges 1. “Artifacts Evaluated – Availability” badge 2. “Artifacts Evaluated – Functional” badge 3. “Artifacts Evaluated - Reusable” badge F. Reasons for the claim: 1. For the “Artifacts Evaluated – Availability” badge our all raw main experimental results and Virtual Image with the working tool are published in Zenodo. 2. For the “Artifacts Evaluated – Functional” badge, we have provided a README file with a Virtual Image of the system. Anyone can read the README and regenerate the results. Also, we have provided detailed workbooks to tally the outcomes. 3. For the “Artifacts Evaluated - Reusable” badge, all requirements for the “Artifacts Evaluated - Functional” are meeting as a prerequisite. Our artifact corresponds to the following characteristics: 3.1. The artifact is highly automated and easy to use. 3.2 It is comprehensively documented, and the documentation describes plausible scenarios on how it could be extended. 3.3 The artifact contains all means necessary such that others can extend it. A tool artifact includes its source code, all not commonly available requirements, and a working description of compiling it. Virtual machines with all requirements are provided. 3.4 The README points to the documentation that is part of the artifact and it describes use case scenarios or details beyond the scope of the paper. Please contact us for any queries: sanghu@nitw.ac.in, rasool.maghareh@huawei.com, arpita@comp.nus.edu.sg, joxan@comp.nus.edu.sg ------------------------------------------------------------------------------------------------------------------------------