Dataset and Code for Root Cause Analysis in Performance Testing: A Grey-Box Evaluation of Large Language Models
Authors/Creators
Description
This repository contains the dataset, source code and context information used in the controlled experiment on Artificial Intelligence-assisted Root Cause Analysis (RCA). The study evaluates the capability of three Large Language Models (GPT-5.5 Instant, Claude Sonnet 4.6, and Gemini 3.5 Flash) to diagnose performance bottlenecks in a Node.js/Express API under load, utilizing a grey-box approach.
The models did not have access to the source code; instead, they performed inference exclusively by cross-referencing network metrics (generated by the k6 tool) with server infrastructure telemetry (CPU and Heap Memory consumption). The scenario documented in this dataset focuses on diagnosing an asynchronous passive wait bottleneck (simulating N+1 queries / blocking I/O).
Context-and-problem.zip: contains all the information given to the evaluators
scripts.rar: contains all the code/scripts used in the experiment
raw-answers.rar: contains all the answers given by the evaluators