Persona-Profiled Agent Predictability: dataset for "Personality-Profiled Virtual Agents Are More Predictable: The Constraint-Entropy Tradeoff for Trustworthy Agent Design"
Description
Session-level data from the predictability experiment comparing persona-profiled and unprofiled LLM agents: trial summaries, test records, behavioral-consistency and persona-fidelity metrics, intermediate-temperature results, and a 36-pair validation of the LLM judge against an independent human rater (scores, judge scores, results and rater instructions).
Dataset accompanying the paper Personality-Profiled Virtual Agents Are More Predictable: The Constraint-Entropy Tradeoff for Trustworthy Agent Design, ACM International Conference on Intelligent Virtual Agents (IVA 2026) (https://doi.org/10.1145/3806774.3827973).
Paper abstract. Effective human-agent cooperation requires that users form mental models of an agent's behavioral tendencies. Yet LLM-based virtual agents are inherently stochastic, undermining the behavioral consistency that mental model formation depends on. We introduce the Constraint-Entropy Tradeoff (CET) model, an information-theoretic design framework that quantifies how persona profiles (behavioral constraints specifying an agent's reasoning style, priorities, and communication patterns) reduce the entropy of a virtual agent's action distribution. The CET model derives that behavioral entropy decays monotonically under constraint strength and identifies an optimal constraint level balancing predictability against flexibility. We validate the framework using a computational testbed with 80 sessions across five conditions, including intermediate-temperature conditions that reveal a threshold effect in the temperature-consistency relationship. Crucially, at the same high temperature, persona-profiled agents recover substantial behavioral consistency compared to unconstrained agents, demonstrating that persona profiles provide independent behavioral constraint beyond temperature reduction. All participants are LLMs; results establish that persona profiles create measurably distinct behavioral patterns, a necessary precondition for human mental model formation, but human validation is needed. We derive domain-specific design guidelines for applications in education, healthcare, and social simulation.
Known limitations.
- Full conversation transcripts were not retained; only per-session metrics are available.
- The 36 validation pairs come from new sessions generated in May 2026, not from the original 80-session corpus.
- Experiment code is not included in this release.
License: Creative Commons Attribution 4.0 International (https://creativecommons.org/licenses/by/4.0/).
Files
croissant.json
Files
(73.2 kB)
| Name | Size | |
|---|---|---|
|
md5:0ad7c8f53030dbb61fb2a76b8420eeaf
|
1.5 kB | Download |
|
md5:7c32edee38f711312ae85894e7689d2d
|
20.7 kB | Preview Download |
|
md5:0f142a318cfb907ab51ba431879471f3
|
378 Bytes | Download |
|
md5:f2f0046fe7cb2294a29ff52585d2dffb
|
44.0 kB | Preview Download |
|
md5:5eed4edb12c24e5743870199029fb671
|
6.6 kB | Preview Download |
Additional details
Related works
- Is identical to
- Dataset: https://huggingface.co/datasets/ai4he/persona-profiled-agents-predictability (URL)
- Is supplement to
- Conference paper: 10.1145/3806774.3827973 (DOI)