QAOA-TCS

QAOA-TCS - Quantum Regression Test Case Selection
This repository contains all the necessary resources to reproduce the results of the QAOA-TCS method.

Dataset Files

The "./datasets" folder contains:

• "sir_programs"

SIR Programs

The "./datasets/sir_programs" folder contains, for each SIR program considered by this project, all the files needed to gather statement coverage, execution costs, and past fault coverage information.

Let's take the "flex" program as an example.

Inside the folder "./datasets/sir_programs/flex" the file "fault-matrix.txt" contains many rows as flex's test cases. Each row is divided by (in this case) 5 columns, representing 5 different versions of the program. Each cell (i,j) contains a binary (0 or 1) value, representing the ability of the i-th test case to spot a fault in the j-th version of the program. This configuration is called a fault matrix and is the resource needed to gain historical fault coverage information.

The folder "./datasets/sir_programs/json_flex" contains a folder for each test case. The generic "ti" folder (i.e. the folder related to the i-th test case of flex) contains the "flexi.gcov.json" file. This JSON file is fundamental to recovering statement coverage and execution costs information since it tells us for each basic block of the system under test if the i-th test case executed/covered it and how many times the i-th test case executed it. This way, we can derive the total statement coverage and execution cost of a single test case.

Source Code Files

generate_sir_program_dataset.py
This file gathers and organizes all the information needed by the four algorithms. Once executed, you will not need to re-read the dataset each time.

MOQ-Pipeline.ipynb

This file contains two main algorithms:

• The three-objectives version of QAOA-TCS
• The Additional Greedy Algorithm

This file implements all the pipelines necessary to run the algorithms, from dataset information gathering to algorithm execution and finally to empirical comparisons.

QAOA-TCS vs SelectQA and Classical Algorithms

The pipeline starts from the analysis of SIR programs and with the comparisons between QAOA-TCS, SelectQA, and the classical algorithms.

DIVGA.m

The file "./MATLAB/DIVGA.m" contains the whole pipeline needed for the execution of the DIVGA algorithm. For simplicity, the statement coverage, execution costs, and fault coverage information already gathered by "./MOQ-Pipeline.ipynb" has been written into .txt files. This way, DIVGA.m just has to read the files to obtain that information, bypassing the actual datasets.

Please note that DIVGA.m is configured to run against one program at a time, so you will have to reconfigure the target and the algorithm parameters each time you change the target program. Also, be sure to reconfigure both the M, N, and gamultiobj routine's parameters (the correct values are described in the paper by Panichella et al. named "Improving Multi-Objective Test Case Selection by Injecting Diversity in Genetic Algorithms").

In line 104, H_size is less than max{N,M} + 1 (so choose max{N,M} + 1). In line 53, the denominator of total_coverage = -length(unique_covered_lines) / 2034; must be changed with the total number of code lines of the target program. Be careful to change the target files for results reporting.

SelectQA.ipynb

In this file, there is the implementation of SelectQA. The difference with the other Jupyter file is that there is no usage of generate_sir_programs_datasets.js.

mo_effectiveness.R

This file contains the code to perform all the statistical analysis between DIV-GA, Additional Greedy, SelectQA, and QAOA-TCS in terms of the number of non-dominated solutions.

mo_efficiency.R

This file contains the code to perform all the statistical analysis between DIV-GA, Additional Greedy, SelectQA, and QAOA-TCS in terms of execution times.

Results Files

The "./results" folder contains all the results obtained by QAOA-TCS, DIV-GA, Additional Greedy, and SelectQA after all the experiment executions. These files are needed to conduct all the empirical evaluations and comparisons between the methods.