Published August 10, 2026 | Version v2.6.0

Kharisara/mer-jit-llm-fgcs: ReplayBench-PG v2.6.0

Authors/Creators

Description

ReplayBench-PG v2.6.0

ReplayBench-PG v2.6.0 is the corrected reproducibility release supporting the manuscript:

ReplayBench-PG: A Layered Evidence Model for Execution-Integrity Validation of Policy-Gated AI Pipelines

Main v2.6.0 corrections

  • Uses the corrected four-policy primary evaluation: risk_proxy, random, always, and never.
  • Separates deterministic configuration identity from concrete execution-instance identity.
  • Uses execution-instance-scoped downstream receipt reconciliation.
  • Retains layered action, row-cardinality, authorization/execution, receipt, record, and configuration integrity evidence.
  • Includes the label-independent evidence comparison:
    • 228 positive evidence units
    • 42 negative evidence units
    • V_primary: 88/228 positive detections
    • V_full: 228/228 positive detections
    • 0/42 negative units flagged by either model
    • 4,754 supported localized events with 0 off-target and 0 missed identifiers
  • Retains the corrected 240-condition primary benchmark, 352-measurement timing study, 54-condition Ray validation, two-region cloud validation, controlled-fault validation, and MetroPT-3 secondary-workload study.
  • Replaces redistribution of the generated MELD replay input with deterministic local reconstruction from provider-obtained MELD annotations.
  • Adds an ordered ID-only MELD provenance manifest and reconstruction verification metadata.

Primary-input reconstruction

MELD source content is not redistributed.

The public release includes:

  • DATA_TERMS.md
  • scripts/prepare_primary_replay_v260.py
  • data_provenance/meld_v260_record_ids.csv
  • paper_outputs/replay_input_v260_manifest.json

Expected reconstruction:

  • rows: 11,351
  • unique source-record IDs: 11,351
  • diagnostic-positive records: 2,609
  • selection-manifest SHA-256: 3ecd5826976393d0c44ae3d59d5d7e7a8b8b6ccd416571dd96b564504f261646
  • canonical reconstructed replay-table SHA-256: 2b46fd5e6887305d2eb43b1f4383e23ed3c1062a4826e1b7a428a10bd8f84f44

Final validation

The frozen artifact passed the master validation suite, including:

  • required artifact checks
  • primary-input reconstruction validation
  • primary benchmark validation
  • timing validation
  • Ray validation
  • execution-integrity validation
  • validator-selectivity validation
  • label-independent validation
  • controlled-fault validation
  • MetroPT-3 validation
  • cloud validation
  • Python compilation
  • full automated test suite

Final repository commit:

0d54c067044da632a37f760d20c61100bfa3d575

Files

Kharisara/mer-jit-llm-fgcs-v2.6.0.zip

Files (542.4 MB)

Name Size
md5:f9fdf503081b8c8f0ad0cc8874aa88fa
542.4 MB Preview Download

Additional details

Related works