Replication Package for "Requirements Traceability Link Recovery via Retrieval-Augmented Generation"
Authors/Creators
Description
This is the replication package for our paper "Requirements Traceability Link Recovery via Retrieval-Augmented Generation".
This package contains the source code, the dataset used in the evaluation, and the results of the evaluation.
Requirements
-
Java JDK 21 + Maven 3
-
OpenAI subscription: API Key
-
Ollama Instance
Structure of this Repository
-
/: Contains the source code of the approach
(Note: The most recent version of the tool can be found at ArDoCo/LiSSA-RATLR) -
/cache: Contains the cache for the evaluation. -
/configs: Contains the configurations used in the evaluation. -
/datasets: Contains the datasets used in the evaluation and the results of the evaluation. -
/results-r2r: Contains the results of the evaluation. -
/experiments: Contains the aggregated results of our experiments, as well as the scripts and results of the statistical tests.
Installation
-
You can use the provided JAR.
-
Alternatively, build the project using Maven:
mvn packageThe JAR will be created in the
targetfolder (e.g.,target/ratlr-*-jar-with-dependencies.jar).
Usage
-
Create a
.envfile based on the provided template. -
Run the JAR from the top-level directory:
java -jar target/ratlr-*-jar-with-dependencies.jar eval -c ./configs/req2req/<<DesiredConfig>>
Results of Evaluation
All results are available in the results-r2r folder. Results are stored as Markdown (MD) files in the respective projects. Each file includes the configuration and the results of the evaluation. Example configuration:
{
"cache_dir" : "./cache/CCHIT",
"gold_standard_configuration" : {
"hasHeader" : true,
"path" : "./datasets/req2req/CCHIT/answer.csv"
},
"source_artifact_provider" : {
"name" : "text",
"args" : {
"artifact_type" : "requirement",
"path" : "./datasets/req2req/CCHIT/high"
}
},
"target_artifact_provider" : {
"name" : "text",
"args" : {
"artifact_type" : "requirement",
"path" : "./datasets/req2req/CCHIT/low"
}
},
"source_preprocessor" : {
"name" : "artifact",
"args" : {}
},
"target_preprocessor" : {
"name" : "artifact",
"args" : {}
},
"embedding_creator" : {
"name" : "openai",
"args" : {
"model" : "text-embedding-3-large"
}
},
"source_store" : {
"name" : "custom",
"args" : {}
},
"target_store" : {
"name" : "custom",
"args" : {
"max_results" : "4"
}
},
"classifier" : {
"name" : "mock",
"args" : {}
},
"result_aggregator" : {
"name" : "any_connection",
"args" : {
"source_granularity" : "0",
"target_granularity" : "0"
}
},
"tracelinkid_postprocessor" : {
"name" : "identity",
"args" : {
"reverse" : "false"
}
}
}
Statistics
-
# TraceLinks (GS): 587
-
# Source Artifacts: 116
-
# Target Artifacts: 1064
Results
-
True Positives: 92
-
False Positives: 372
-
False Negatives: 495
-
Precision: 0.1983
-
Recall: 0.1567
-
F1 Score: 0.1751
Files
ArDoCo/ReplicationPackage-REFSQ25_Requirements-TLR-via-RAG-v1.0.0.zip
Additional details
Related works
- Is supplement to
- Software: https://github.com/ArDoCo/Replication-Package-REFSQ25_Requirements-TLR-via-RAG/tree/v1.0.0 (URL)