Published October 24, 2026
| Version v1
Preprint
Open
Human inter-rater reliability of retrospective neuro-type classification: a Registered Report Stage 1 protocol (stratified n=200, blinded multiple raters)
Description
Stage 1 Registered Report for a blinded human inter-rater reliability study. A stratified random sample of 200 historical figures plus 30 gold-standard items is independently coded by at least three blinded raters with the six-dimensional rubric. Primary endpoint: six-category Fleiss kappa of at least 0.70 with bootstrap 95% lower bound at least 0.60 (three-category ND-subtype kappa at least 0.80); Gwet AC1 reported as a robustness check. A first-round kappa below 0.50 triggers exactly one rubric revision; persistent failure below 0.50 constitutes person-level falsification of the coding scheme. LLM-based ratings (observed agreement up to ~0.79) are treated as auxiliary and cannot replace human reliability. The protocol is registered before raters are recruited or any results are seen; requires IRB review.