LLM Judge Bias (LJB) in Self-Involved Critique: Identity-Conditioned Defensiveness in LLM Evaluation

Bias in LLM-as-judge evaluation is usually studied in observer settings, where the model evaluates text written by others. This paper asks whether analogous identity labels behave differently when the model itself becomes the target of criticism. Across two studies, we compare an observer-frame label manipulation (MIB, N=1,983) with a self-involved adversarial critique setting (AEB, N=3,609). In the observer frame, author labels produced no measurable score differences (F=0.042; TOST 10/10 equivalent). In the self-involved frame, the analogous label-swap produced reliable differences in defensive responses. A broader source-level analysis (Phase A, six conditions) found that critic identity formed a two-cluster pattern rather than a monotonic gradient: competitor and peer critics elicited higher defensiveness than anonymous, human-expert, and user critics. A focused confirmation on a separately generated response set (Phase B, three conditions) reproduced the core categorical contrast: a named competing-model critic elicited greater defensiveness than anonymous and human-expert critics (η²=.028, p<.001). Both phases hold the critique text fixed and vary only the critic label. We measure defensiveness with the Defensive Response Index (DRI), a protocol-bound 0–1 composite validated against development-anchor labels, a five-LLM judge ensemble, and two external human raters, and robust to bootstrap resampling, random weighting, length control, and scale collapse. Defensiveness appears not as overt hostility but as subtle, consistent shifts in rebuttal density and score defense. These findings suggest identity-conditioned defensiveness: author labels are inert in observer-frame evaluation, whereas critic labels become design-relevant when LLMs respond to criticism of their own outputs.

Note: This preprint contains the main paper only. Supplementary appendices have been prepared as separate peer-review materials and are not included in this public version.