RUHSAT-Bench: a 473-claim abstention benchmark over Turkish construction and occupational-safety regulation — data, code and pre-registration

Release archive for the paper Measuring abstention, not accuracy: a re-qualification benchmark for language-model decision support on Turkish construction and occupational-safety regulation.

RUHSAT-Bench is a set of 473 true/false claims over six frozen Turkish regulatory documents (Laws 3194, 4708 and 6331, the Building Inspection Implementation Regulation, the OHS Risk Assessment Regulation, and the Turkish Building Earthquake Code TBDY 2018). Each claim is asked under two conditions: one permitting an explicit "not sure" and one forcing a binary verdict, so that abstention behaviour is measured rather than assumed.

The archive contains the source documents with checksums, three parsed corpora, the claim set in three gold-label versions, the blinded expert-audit workbooks including the negative second-pass result, the pre-registration and its eight annexes, every run record, the scoring and analysis code, and a number sheet mapping every figure in the paper to the file and script that produced it.

Two limits are stated in the repository README: the development repository's version history was lost and the repository was reconstructed from dated hand-over packages, so pre-registration timing rests on document dates rather than commit timestamps; and the expert coders are identified only by role.