When Metrics Lie: A Protocol for Uncovering Flawed Human-Likeness in Team Shooters

Before an agent can be trained to imitate human team tactics in a competitive shooter, one must be able to decide whether it has succeeded. The candidate behavioral metrics we examine (traded death rate, flash assist rate, save rate, drop rate and seven others in common use) have generally been introduced without an explicit test of discriminative validity, and we found none in our search. We introduce DISCERN, a screen built on a known-groups contrast: within-series pairing holds the shared match-level factors (map, event, patch, score context) identical across the two sides by construction; every metric must register a face-value directional prediction before any test is run, with direction mismatch kept as a verdict of its own; and five self-checks cover the pipeline, the free parameters and agreement with an external implementation.

Applied to eleven candidate metrics over 1,558 professional demo files (782 series), with tiers taken from the HLTV ranking in force on the date of each series (368 mixed-tier series), DISCERN retains four. Two are substantive behavioral metrics (traded death rate and drop rate), and both still separate the tiers when the recurrence of teams across series is accounted for. Six show no separation detectable at this design, whose minimum detectable effect is about r = 0.18. One metric, trade kill rate, is significant in the direction opposite to its construct; the reversal survives every labeling of tiers and the team-aware analysis, and we trace it to the opportunity structure of its denominator, which is shared exactly with the identically named metric published by a public statistics site. In kill situations where a line of sight certainly existed, the dataset's visibility field fires in only 39.5% of cases, which limits every metric built on it. Throughout, the screen is on team standard, not on human-likeness; the two are linked by an assumption we make explicit rather than test.