There is a newer version of the record available.

Published August 10, 2025 | Version 1.0.0

AI-augmented Cybersecurity Requirements Generation using LLMs | Reproducible Research Package

  • 1. ROR icon Universidad Politécnica de Madrid

Description

AI-augmented Cybersecurity Requirements Generation using LLMs | Reproducible Research Package

This repository accompanies the paper “Experimental Evaluation of AI-Augmented Cybersecurity Requirements Generation Leveraging LLMs’ Capabilities.” It contains every script, dataset, prompt template and result needed to fully reproduce our empirical study.

Research Description

This project investigates the practical use of state‑of‑the‑art Large Language Models (LLMs) to transform high‑level, standard‑driven cyber‑security controls into concrete, system‑specific requirements. Using a synthetic yet industrially plausible case study—AI4I4, an IoT‑enabled automotive logistics platform—we benchmark thirteen frontier models (GPT‑4, LLaMa 3, Mistral, QWen, etc.), representing tge state of the art as of September 2024, across four prompting pipelines and three temperature regimes.

Key contributions include:

1. **Annotated benchmark** of 54 ISO‑27002 clauses with placeholder semantics suitable for automatic instantiation.
2. **LangChain pipelines** that decompose the task into applicability filtering, domain‑element search, requirement generation, and JSON formatting.
3. **Comprehensive evaluation** of accuracy (precision, recall, F2), creativity (F2‑synthetic), and consistency (Jaccard overlap across runs).
4. **Prompt library** enumerating >180 templates, showing how subtle changes in instruction design affect hallucination rate and coverage.

The artefacts and scripts below allow full replication—from raw prompts to final figures—on any infrastructure with access to the referenced models.
 
For more information on the repository structure, reproducibility, licensing, and contact details, please refer to the README.

Files

ai_requirements_generation_rr.zip

Files (930.3 kB)

Name Size Download all
md5:0bc2250b6742b5c57e0abf7453a9f086
930.3 kB Preview Download

Additional details

Funding

Ministerio de Ciencia, Innovación y Universidades
PRESECREL Modelos y plataformas para sistema informáticos industriales predecibles, seguros y confiables PID2021-124502OB-C43

Dates

Available
2025-08-10
Initial public release

Software

Repository URL
https://github.com/STRAST-UPM/ai_requirements_generation_rr
Programming language
Python
Development Status
Active