When Metrics Lie: A Protocol for Uncovering Flawed Human-Likeness in Team Shooters

Before an agent can be trained to imitate human team tactics in a competitive shooter, one must be
able to decide whether it has succeeded. The candidate behavioral metrics we examine—traded

death rate, flash assist rate, save rate, drop rate and seven others in common use—have gener-
ally been introduced without an explicit test of discriminative validity, and we found none in our

search. We introduce DISCERN, a screen built on a known-groups contrast: within-series pair-
ing holds the shared match-level factors—map, event, patch, score context—identical across the

two sides by construction; every metric must register a face-value directional prediction before
any test is run, with direction mismatch kept as a verdict of its own; and five self-checks cover
the pipeline, the free parameters and agreement with an external implementation. Applied to
eleven candidate metrics over 1,558 professional demo files (782 series, 324 of them mixed-tier),

DISCERN retains three—of which one is mechanically confounded, leaving two substantive be-
havioral metrics. Six show no detectable separation at this contrast, and two are significant in

the direction opposite to their construct; for trade kill rate we trace the reversal to the opportu-
nity structure of its denominator, a denominator shared exactly with the identically named metric

published by a public statistics site. All three metrics related to perceptual asymmetry fail, for a
traceable reason: in kill situations where a line of sight certainly existed, the dataset’s visibility
field fires in only 39.5% of cases. Verdicts are invariant over trade windows from 3 to 7 seconds,
and one retained metric reproduces public per-player statistics for 15 players (weighted r = 0.90,

detection rate 0.83). We state the negative result at the scope the evidence supports: percep-
tual asymmetry, as operationalized through this dataset’s parsed visibility field, cannot be

measured reliably from that field. Under the current data representation and the low-cost ob-
servational pipeline we proposed, that closes the cheap route and makes a controlled environment

a practical requirement—not the only conceivable one, since geometric reconstruction from raw
demos remains untried. Throughout, the screen is on team standard, not on human-likeness; the
two are linked by an assumption we make explicit rather than test.