LLM Judge Bias (LJB) in Self-Involved Critique: Identity-Conditioned Defensiveness in LLM Evaluation

Bias in LLM-as-judge evaluation is usually studied in observer settings, where the model evaluates text written by others. This paper asks whether analogous identity labels behave differently when the model itself becomes the target of criticism. Across two studies, we compare an observer-frame label manipulation (MIB, N=1,983) with a self-involved adversarial critique setting (AEB, N=3,609). In the observer frame, author labels produced no measurable score differences (F=0.042; TOST 10/10 equivalent). In the self-involved frame, the analogous label-swap produced reliable differences in defensive responses. A broader source-level analysis (Phase A, six conditions) found that critic identity formed a two-cluster pattern rather than a monotonic gradient: competitor and peer critics elicited higher defensiveness than anonymous, human-expert, and user critics. A focused confirmation on a separately generated response set (Phase B, three conditions) reproduced the core categorical contrast: a named competing-model critic elicited greater defensiveness than anonymous and human-expert critics (η²=.028, p<.001). Both phases hold the critique text fixed and vary only the critic label. We measure defensiveness with the Defensive Response Index (DRI), a protocol-bound 0–1 composite validated against development-anchor labels and showing convergence with a five-LLM judge ensemble and two external human raters, and robust to bootstrap resampling, random weighting, length control, and scale collapse. Defensiveness appears not as overt hostility but as subtle, consistent shifts in rebuttal density and score defense. These findings suggest identity-conditioned defensiveness: author labels are inert in observer-frame evaluation, whereas critic labels become design-relevant when LLMs respond to criticism of their own outputs. Because the two studies differ in task and dependent variable, we treat this contrast as a cross-study asymmetry rather than a single factorial test of frame. We interpret the pattern as a source-conditioned response prior, offered as a post hoc account and not a claim of human-like psychology.

Version note. This public preprint contains the main paper only. Appendix references in the paper refer to supplementary materials prepared separately for peer review and are not included in this public version.