Experimental Dataset for Evaluating Security and Inclusivity in LLM-Generated Code

v1.0.1 - Initial Public Release

Experimental Dataset for Evaluating Security and Inclusivity in LLM-Generated Code

This release contains the complete dataset and replication materials from our controlled experiment examining the relationship between security and inclusivity in Large Language Model (LLM)-generated authentication code.

The data was generated and collected during September-October 2025.

What's Included

Input Data

• 3 action prompts with varying inclusivity specifications (None → Moderate → Detailed)
• 4 system prompts for the multi-agent code generation pipeline

Generated Artifacts

• 3 complete TypeScript password recovery applications generated by GPT-5
• Per-iteration code snapshots, task decompositions, and evaluation reports
• Token usage metrics and cost breakdowns

Evaluation Data

• 15-item security rubric based on OWASP Top 10 (2025)
• 15-item inclusivity rubric based on 5 cognitive dimensions
• LLM evaluation results from 5 models (GPT-5, Claude Sonnet 4.5, Gemini 2.5 Pro, Mistral Medium 3.1, DeepSeek 3.2)
• Human expert evaluation results (n=13: 8 security experts, 5 inclusivity experts)
• Survey questionnaires used for data collection (PDF)

Replication Materials

• Complete Python pipeline for multi-agent code generation
• Configuration templates and documentation

Quick Start

git clone https://github.com/Japskua/paper_llm_for_secure_inclusive_auth.git
cd paper_llm_for_secure_inclusive_auth

# Review generated code
ls workspace/password_recovery_health/

# Review evaluation results
ls final_evaluations/results/
ls human_evaluations/