arananet/contam-bench: v0.3.2 - Scoring audit and independent-review preparation
Description
Scope
This prerelease updates research tooling, documentation, and preparation for independent review. It does not introduce new experimental results or establish scientific validity.
Changes
- Document canonical regex mention and Markdown-formatting limitations, and whole-response cue limitations in the auxiliary assertion classifier. Characterization tests expose these behaviors; the auxiliary heuristic has not been repaired or validated.
- Verify exact coverage of all 52 pending review packets, response and rubric preservation, exclusion of hidden metadata fields, deterministic generation, and unchanged evidence files.
- Add neutral reviewer instructions, recruitment guidance, abstention handling, and independent submission procedures.
- Calibrate manuscript and README claims to the pilot evidence and distinguish reproducibility checks from independent validation.
- Add release checks for the extracted paper source archive and read-only provenance reporting.
Verification
make release-check passed for commit 52126c7f5905bc43e870614630d5de9866d6da16: OpenSpec checks, 90 tests, a separate rerun of the 25-test schema subset, and successful compilation of the extracted paper archive. The schema rerun is not 25 additional unique tests.
Evidence and limitations
- Frozen evidence, canonical machine verdicts, and earlier tags remain unchanged.
- The repeated run still has 350 scored rounds, including 52 unresolved rounds.
- There are zero human adjudications and zero two-reviewer consensuses in the frozen queue.
- No new model calls, experiments, or auxiliary rescoring were performed for this release.
- Blinded human assessment, a prospective assessment of machine-resolved outcomes, and external reproduction remain pending.
- Passing engineering checks does not establish scorer accuracy, mitigation efficacy, peer review, or publication acceptance.
No reviewer-kit archive is attached. Reviewers should receive the neutral kit privately and avoid consulting this release, the repository, or machine results before completing their assessments.
See the scoring audit and adjudication operations for coordinator documentation.
Files
arananet/contam-bench-v0.3.2.zip
Additional details
Related works
- Is supplement to
- Software: https://github.com/arananet/contam-bench/tree/v0.3.2 (URL)
Software
- Repository URL
- https://github.com/arananet/contam-bench