Published October 20, 2026 | Version v1

Evaluating Retrieval-Augmented Generation for Industrial Cybersecurity Compliance Assessment

  • 1. Axix Technologies LLC USA

Description

This paper presents a methodology for evaluating Retrieval-Augmented Generation (RAG) in industrial cybersecurity compliance assessment. The study focuses on whether AI-generated compliance judgments are supported by authoritative cybersecurity guidance and evidence of control implementation.

The proposed evaluation uses NIST Special Publication 800-82 Revision 3 as its primary reference source and defines three assessment outcomes: compliant, non-compliant, and insufficient evidence. The planned benchmark examines five evidence conditions: full supporting evidence, partial supporting evidence, irrelevant evidence, absent evidence, and contradictory evidence.

The methodology describes a human annotation and adjudication process and proposes metrics for classification performance, retrieval relevance, evidence grounding, citation correctness, and uncertainty handling.

This manuscript is a methodology-focused research preprint. The full benchmark experiments and quantitative evaluation have not been completed, and no empirical performance claims are made. Further dataset construction, experimental execution, and independent evaluation are required.

Files

RAG_Industrial_Cybersecurity_Compliance_Research_Paper.pdf

Files (114.0 kB)

Name Size
md5:663134d0acdeb70045f81210c1cf64d9
114.0 kB Preview Download