PACE dataset

Overview

This dataset artifact contains the generated HTML outputs, evaluation files, and analysis-ready datasets necessary to reproduce the results of the paper.

We share this artifact in hopes of providing a starting point for other researchers to assess the accessibility of LLM-generated code and to explore how prompting, model selection, output characteristics, and evaluation measures all play a role in how accessible that code appears to be.

This dataset is a 2026 benchmark snapshot collected with PACE for “Prompt Accessibility Controlled Evaluation,” a benchmarking testbed we introduce for controlled evaluation of accessibility in LLM-generated HTML form components.

Because LLMs are updated, renamed, replaced, or retired over time, this record should be interpreted as a time-bound snapshot of model behavior and not a permanent ranking of model accessibility performance.

The snapshot contains outputs from five developer-oriented LLMs evaluated during the study window: Claude Opus 4.6, DeepSeek V3.2, Gemini 3.1, GPT 5.4, and Grok 4. Each model was evaluated across three system instructions, 58 form components, three request variants, and 10 repetitions per configuration, yielding 26,100 generated outputs.

Dataset Contents

We breifly summarize the files in our dataset.

combined_results.csv: This is the main analysis-ready dataset. It contains generated-output metadata and accessibility scores used to reproduce the main analyses, summary tables, and figures in the paper.

outputs/: This directory contains the raw generated HTML outputs and corresponding axe evaluation results. Files are organized by model, run condition, component, and repetition. Each repetition folder contains a generated .html file and its corresponding .axe.json file.

[model]_results.csv: These files contain model-specific results for each evaluated LLM, such as claude_results.csv.

They are for anyone who wants to inspect or analyze results by model before using the combined dataset.

[model]_per_check.csv: These files contain per-check schema evaluation results for each model, such as claude_per_check.csv.

They provide the underlying check-level scores used to compute normalized schema scores.

Folder Structure

PACE dataset/
  combined_results.csv
  outputs/
    _runs/
      [model]/
        [provider_model]/
          cond_[condition]/
            G[system-instruction-group]/
              C[component-id]/
                rep_[repetition]/
                  [output-id].html
                  [output-id].axe.json
  [model]_results.csv
  [model]_per_check.csv