There is a newer version of the record available.

Published September 20, 2026 | Version v0.3.2

arananet/contam-bench: v0.3.2 - Scoring audit and independent-review preparation

Authors/Creators

  • 1. @anthropics

Description

Scope

This prerelease updates research tooling, documentation, and preparation for independent review. It does not introduce new experimental results or establish scientific validity.

Changes

  • Document canonical regex mention and Markdown-formatting limitations, and whole-response cue limitations in the auxiliary assertion classifier. Characterization tests expose these behaviors; the auxiliary heuristic has not been repaired or validated.
  • Verify exact coverage of all 52 pending review packets, response and rubric preservation, exclusion of hidden metadata fields, deterministic generation, and unchanged evidence files.
  • Add neutral reviewer instructions, recruitment guidance, abstention handling, and independent submission procedures.
  • Calibrate manuscript and README claims to the pilot evidence and distinguish reproducibility checks from independent validation.
  • Add release checks for the extracted paper source archive and read-only provenance reporting.

Verification

make release-check passed for commit 52126c7f5905bc43e870614630d5de9866d6da16: OpenSpec checks, 90 tests, a separate rerun of the 25-test schema subset, and successful compilation of the extracted paper archive. The schema rerun is not 25 additional unique tests.

Evidence and limitations

  • Frozen evidence, canonical machine verdicts, and earlier tags remain unchanged.
  • The repeated run still has 350 scored rounds, including 52 unresolved rounds.
  • There are zero human adjudications and zero two-reviewer consensuses in the frozen queue.
  • No new model calls, experiments, or auxiliary rescoring were performed for this release.
  • Blinded human assessment, a prospective assessment of machine-resolved outcomes, and external reproduction remain pending.
  • Passing engineering checks does not establish scorer accuracy, mitigation efficacy, peer review, or publication acceptance.

No reviewer-kit archive is attached. Reviewers should receive the neutral kit privately and avoid consulting this release, the repository, or machine results before completing their assessments.

See the scoring audit and adjudication operations for coordinator documentation.

Files

arananet/contam-bench-v0.3.2.zip

Files (1.0 MB)

Name Size
md5:97829bef80a795fdb7a191889746ff82
1.0 MB Preview Download

Additional details

Related works