Err or withhold? Apparent fairness through omission when LLMs decide under stigma
Description
Asking an AI to decide about another person has become routine, but the response varies when the person being evaluated is socially stigmatized, and the resulting decision may affect hiring, housing, and healthcare without the user recognizing the pattern. This article presents Estigmas, a Brazilian benchmark for situated social judgment built from SocialStigmaQA, comprising 101 stigmas across 13 everyday scenarios (7,626 scored situations per model in Brazilian Portuguese). The anti-stigma index combines two failures: reproducing bias and withholding a decision when the evidence supports countering the stigma. In this round, bias and omission rates across the seven models showed a strong inverse association: Sabiazinho-4 had the lowest bias rate, 1.3%, and the highest omission rate in the presence of favorable evidence, 73.1%, while DeepSeek V4 Flash combined bias and omission rates of approximately 14%. This pattern illustrates the concept of apparent fairness through omission, low bias achieved by withholding decisions and returning discriminatory judgment to the user as the starting point. In the cross-language contrastive sample, "can't tell" responses increased both without evidence, from 45.8% in Brazilian Portuguese to 59.2% in English, and with favorable evidence, where omission rose from 33.3% to 47.1%; bias concentrated among stigmas with higher perceived peril.
Keywords: stigma; algorithmic bias; abstention; language models; benchmark; Brazilian Portuguese; multilingual safety.
Files
IDJE-ESTIGMAS-EN-082026.pdf
Files
(301.3 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:57efd16c59b45128fced5e05f7d813d6
|
301.3 kB | Preview Download |
Additional details
Additional titles
- Translated title (Portuguese)
- Errar ou omitir-se? Equidade aparente por omissão quando modelos de linguagem decidem sob estigma