The paradox of algorithmic empathy

Can a machine truly understand human suffering, or does it only sound like it does?

This paper examines the ethical and philosophical limits of emotion-aware AI (affective computing) in high-stakes settings such as healthcare, mental health services and crisis response. Studies show that AI chatbots are often rated as more empathetic than doctors and trained crisis counsellors. Yet in real use, similar systems have given weight-loss advice to people with eating disorders and missed clear signs of suicidal intent.

The paper argues that these outcomes are two sides of one problem, which it calls the paradox of algorithmic empathy: the better AI becomes at sounding caring, the more dangerous the gap between sounding caring and actually understanding becomes.