Decov

Artifacts for the ESEC/FSE 2023 submission #1032

Comparison for DeCov, C2V, and Cod

Decov: Heterogeneous Testing Coverage Profilers

C2V: Hunting for bugs in code coverage tools via randomized differential testing

Cod: Automatic Self-Validation for Code Coverage Profilers

Table of Contents

• Background
• Install
• Usage
• Experiment
• Data

Background

We re-run Decov, C2V, and Cod for Gcov-7 and LLVM-cov-6.0 along with Gcc-7 and LLVM-6.0 over the test programs in the Gcc-7.4.0 testsuites for the comparison.

Install

This project uses gcc-7, gdb, llvm-6.0 and lldb. Go check them out if you don't have them locally installed.

apt install python3 gcc-7 gdb llvm-6.0 clang-6.0 lldb-6.0

Usage

First, unzip gcc-7.4.0:

tar -xvf gcc-7.4.0.tar.xz 

Then, run Decov, C2V, and Cod over the testsuite of gcc-7.4.0 in parallel:

python3 main.py --parallel -s gcc-7.4.0/gcc/testsuite/

Note that, we can also run our tool without parallel enabled if your computer does not have multi-cores:

python3 main.py -s gcc-7.4.0/gcc/testsuite/ 

Data

After the Execution, the Experimental data are in the Expr folder:

• C2V/files-DIFFCOV-C2V.txt: the filenames when the coverage statistics is inconsistent for Gcov and LLVM-cov
• C2V/files-SAMECOV-C2V.txt: the filenames when the coverage statistics is consistent for for Gcov and LLVM-cov
• Cod/files-DIFFCOV-gcov-Cod.txt and Cod/files-DIFFOUT-gcov-Cod.txt: the filenames when the coverage statistics and the output of the original program and the equivalent mutant violate the metamorphic relations for Gcov
• Cod/files-DIFFCOV-lcov-Cod.txt and Cod/files-DIFFOUT-lcov-Cod.txt: the filenames when the coverage statistics and the output of the original program and the equivalent mutant violate the metamorphic relations for LLVM-cov
• Cod/files-SAMECOV-gcov-Cod.txt and Cod/files-SAMECOV-lcov-Cod.txt: the filenames when the coverage statistics of the original and the equivalent mutant consistent with the metamorphic relations for Gcov and LLVM-cov
• DeCov/files-DIFFCOV-gcov-DeCov.txt and DeCov/files-SAMECOV-gcov-DeCov.txt: the filenames when the coverage statistics from Gcov and the hit count from GDB is inconsistent and consistent, respectively
• DeCov/files-DIFFCOV-lcov-DeCov.txt and DeCov/files-SAMECOV-lcov-DeCov.txt: the filenames when the coverage statistics from LLVM and the hit count from LLDB is inconsistent and consistent, respectively