Plot-to-Rule Inference: Converting Visual Performance Narratives into Modular CMA-ES Configuration Policies

README — Folder structure

This Zenodo record contains all prompts and LLM outputs used in the experiments for the paper:

“Plot-to-Rule Inference: Converting Visual Performance Narratives into Modular CMA-ES Configuration Policies”

The repository is organized as a sequential pipeline. Each numbered folder corresponds to one experimental step; most steps follow the same internal structure:

<step-folder>/
  Prompt/    # prompt templates + static inputs (images/txt)
  Results/   # LLM outputs produced from the prompts

Several steps also depend on outputs from previous steps (e.g., knowledge bases, JSON specifications, rulebooks, inferred configurations).

Top-level folders (pipeline steps)

1-Plot to text knowledge bases/

Creates the textual knowledge base from visual diagnostics (marginal importance plots + interaction heatmaps), separately for each problem and dimensionality.

• Prompt/: prompt templates + diagnostic plot bundles.

• Results/: generated knowledge-base texts (used as inputs in later steps).

2-Function codebook - from text to configuration/

Transforms the textual knowledge into a structured function-level “codebook” (e.g., specification/JSON-style representation of what to enable/disable).

• Depends on: Step 1 outputs.

• Results/: structured descriptors used for configuration consolidation and rule induction.

3-Leave-one-out rulebooks/

Generates leave-one-problem-out (LOPO) rulebooks from the extracted specifications.

• Depends on: Step 2 outputs.

• Results/: one rulebook per held-out function (ordered rules; priority matters).

4-Rules to configuration for LOPO/

Applies the LOPO rulebooks to infer configurations for the held-out problem.

• Depends on: Step 3 rulebooks (+ problem descriptors).

• Results/: inferred configurations per function.

5 - Evaluation of LOPO configurations/

Evaluates LOPO-inferred configurations using the fixed-budget performance archive.

• Depends on: Step 4 inferred configurations.

• Results/: per-function deviations (e.g., Δ) and evaluation summaries.

6 - From function to configuration/

Direct function-to-configuration (F2C) reconstruction: consolidates extracted knowledge into a canonical executable configuration (without LOPO rules).

• Depends on: Step 2 outputs (and/or Step 1 knowledge base, depending on the prompt design).

• Results/: reconstructed canonical configurations per function.

7 - Evaluation of F2C/

Evaluates the F2C configurations using the same fixed-budget reference archive.

• Depends on: Step 6 inferred configurations.

• Results/: per-function deviations (Δ) and evaluation summaries.

8 - Baseline/

Baseline runs where the LLM receives only high-level problem properties + module options, without using the constructed knowledge base.

• Results/: baseline configurations and evaluations for comparison.

Aggregated results (plots+csv)/

Final paper-ready aggregates:

• CSV files (per-run/per-model summaries, Δ tables, statistics)

• Plots (heatmaps, robustness plots, CD diagrams, etc.)

Results organization: dimensionality, model, repetitions

All LLM outputs are stored in Results/ and are grouped as follows:

• Dimensionality: 5D/ and 30D/

• Model: chatgpt/, claude/, gemini/

• Repetitions: three runs (e.g., run1/, run2/, run3/)

Important: The experiments were executed three times per model. The ordering of runs matters, because each run corresponds to a specific prompt execution order and therefore to a specific mapping from input → output. Use the run index to match produced outputs back to the exact prompt instance.

Prompt folder contents

Each Prompt/ folder contains:

• Prompt templates (the exact text used to query the model)

• Static inputs required by that step (e.g., images, plot bundles, text files)

• In later steps, prompts may also require artifacts from previous steps (e.g., knowledge base texts, extracted JSON, rulebooks). These dependencies are documented by the step order above.

Quick navigation tip

If you only want the final artifacts used in the paper:

• start with Aggregated results (plots+csv)/

If you want to trace a single function end-to-end:

• follow the step numbers from 1 → 7 (or 1 → 5 for LOPO), within the same dimension / model / run path.