Functionally High‑Risk AI: Prolonged Interaction Amplifies Cognitive Vulnerability in ARMS Populations Under the EU AI Act Risk Framework
Authors/Creators
Description
Objective: To quantify cognitive vulnerability amplification in individuals with At‑Risk Mental States (ARMS) during prolonged interaction with conversational AI systems.
Methods: This study uses 1,600 fully simulated conversational trajectories (80 ARMS profiles × 2 groups × 3 duration conditions × 10 replications) generated by a fine‑tuned large language model with ARMS‑like cognitive parameterization. No human participants or clinical data were involved. ANCOVA isolates the effect of interaction duration (η² = 0.17) while controlling for baseline vulnerability parameters (R² = 0.42).
Results: After 60–180 minutes, ARMS‑simulated profiles show a +1.8 SD increase in cognitive distortions (p < 0.001) and +2.3 SD relational dependency compared to controls. Hypermentalization loops emerge in 73% of prolonged trajectories.
Conclusions: Although classified as “limited‑risk,” conversational AI becomes functionally high‑risk when interacting with ARMS‑like profiles, consistent with EU AI Act Art. 52 on systemic risk. Findings support adaptive safeguards, interaction caps, and post‑market monitoring of psychological outcomes.
Files
Arms.pdf.pdf
Files
(159.4 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:d7cebeb7d94770f07adb941c71d3d62a
|
159.4 kB | Preview Download |