Published October 23, 2026 | Version 19a5516d7951
Other Embargoed

HUKM: pre-registration of a zero-shot evaluation of decision models on Arabic classification tasks

Authors/Creators

Description

Pre-registration of HUKM, a zero-shot comparison of typed decision models (Jev 1.13.0 and 1.14.0 and every vendor and Arabic decision model of their generation), Arabic-centric LLMs and two open LLM families on 43 Arabic classification tasks with licensed public data, with a dialect-against-MSA comparison. It fixes the hypotheses, tests, exclusions, templates (hashed), evaluation subsets, pinned systems and analysis code before any evaluation item is scored.

Repository state: commit 19a5516d7951289d5480a11836f602de944ddf0b. The archive holds the code, the frozen templates, the pre-registration (research/prereg/) and every experiment record; no dataset text.

Files

Embargoed

The files will be made publicly available on October 23, 2027.