Published January 23, 2025 | Version v1
Dataset Open

Data of Differential Testing of Concurrent Classes - ICST 2025

Authors/Creators

  • 1. ROR icon University of Auckland
  • 2. ROR icon Hong Kong University of Science and Technology

Description

ConDiff Artifact - ICST 2025

Differential Testing of Concurrent Classes

Valerio Terragni 
University of Auckland, New Zealand  
v.terragni@auckland.ac.nz
https://valerio-terragni.github.io/

Shing-Chi Cheung 
The Hong Kong University of Science and Technology, Hong Kong  

Accepted at ICST 2025


@inproceedings{Terragni2025ConDiff,
  author    = {Valerio Terragni and Shing-Chi Cheung},
  title     = {Differential Testing of Concurrent Classes},
  booktitle = {The 18th IEEE International Conference on Software Testing, Verification and Validation (ICST)},
  year      = {2025},
  publisher = {IEEE},
}

Subjects and Results

Folders Overview
The following folders contain all the program revisions used in our experiments:

- benchmarksC-B
- benchmarksC-BR100  
- benchmarksC-BR50  
- benchmarksC-BR25

Each folder includes two program versions:
- V1: The version without the fault.
- V2: The faulty version.

Results Folder
The `results` folder contains the outputs of ConDiff and the Baseline. The files are organized as follows:

- Files starting with `true_`: Results of ConDiff with the filtering phase enabled.
- Files starting with `false_`: Results of the Baseline, i.e., the variant of ConDiff without the filtering phase enabled.

Each file includes the results of 10 runs for each subject.

---

Results Details
The results include the detected behavioral differences (if any) and the following metrics (all time measurements are in milliseconds):

- SR: Success Rate  
- DRT: Difference Revealing Time  
- CT: Number of concurrent tests generated  
- CTB: Number of concurrent tests analyzed by the behavioral checker  
- CTPRUNED: Number of pruned tests  
- PREFIXPRUNED: Number of pruned prefixes that lead to sequential behavioral differences  
- SUFFIXPRUNED: Number of pruned suffixes that lead to sequential behavioral differences  
- CFP: Number of concurrent function pairs containing at least one changed method  

Time Measurements
- TIME_SCHANGE: Time to identify methods that are changed across revisions  
- TIME_GEN: Time to generate the tests  
- TIME_DIFFSEQ: Time for sequential behavioral checking  
- TIME_DIFFCON: Time for concurrent behavioral checking  
- TIME_CIA: Time for change impact analysis  

Files

data-condiff-icst2025.zip

Files (9.2 MB)

Name Size Download all
md5:86e347b78ceec3e0d2969a390e80fbb0
9.2 MB Preview Download