An Exploratory Study of Inherited Bias in AI-Assisted Evaluation: Thirteen Corrections, Zero Content Changes

This study reports an exploratory single-case study in which 82 creative works were submitted to an AI (Claude Opus 4.5) as anonymous text for evaluation. During the evaluation, 13 bias episodes were identified. Each time the human corrector logically challenged a deduction rationale grounded in information external to content, the AI revised its evaluation criteria and the score shifted—without any change to the content itself (1,790 to 2,255 out of 2,400, +25.9% as a descriptive within-session shift).

 

The biases were grouped into five provisional categories: (1) reputation/authority, (2) diffusion/market, (3) format/medium, (4) tool/authorship, and (5) action/realization. Five preliminary design implications are proposed, including pre-evaluation debiasing prompts, iterative debiasing protocols, multi-model cross-validation, standardized human corrector roles, and bias audit reports.

 

This study proposes that AI blind evaluation becomes effective not as a one-time blinding technique but when combined with an iterative correction attitude. The present design cannot fully distinguish bias reduction from sycophantic agreement or rubric renegotiation. Replication with independent evaluators and multiple AI models is needed.

 

Keywords: AI bias, blind evaluation, halo effect, bias inheritance, human-AI collaborative assessment, cognitive bias correction, dialogic correction