Artificial Advice, Real Risks: Are LLM Personas Ready for Stakeholder Roles in Climate Innovation Evaluation?

Achieving net zero emissions and limiting warming to well below 2°C requires the widespread deployment of game-changing innovations (GCIs) — technologies capable of unlocking deep decarbonisation in hard-to-abate sectors such as industry, construction, and aviation. Incorporating realistic assumptions for such innovations into mitigation pathways derived with integrated assessment models (IAMs) remains challenging given the significant uncertainties around costs, materialisation risks, and upscaling timelines. Although stakeholder engagement featuring expert judgement can improve the real-world relevance of these pathways, it is costly, logistically demanding, and requires sustained commitment across diverse technological and geographical contexts. Large Language Models (LLMs) have recently emerged as a potentially lower-cost means of supplementing such engagement through human-like generated personas, though concerns over systematic bias persist. This study compares previously elicited perceptions of 260 global experts in energy, climate, and sustainability with those of LLM-generated personas across 14 GCIs, evaluated on three criteria: mitigation potential, expected time of adoption, and risk of delay. Expert responses were replicated by personas generated using the thinking functions of the widely established LLMs ChatGPT, Gemini, and Claude, each prompted with the occupational, geographical, and gender characteristics of their human counterparts, with all responses aggregated and ranked using the APOLLO group decision-making framework. Results reveal meaningful divergences: LLMs exhibited considerably greater evaluative spread than experts, whose assessments were more balanced. While convergence was observed for certain technologies—notably next-generation energy storage, ocean liming, and hyperloops—LLMs systematically favoured bioenergy technologies, whereas experts prioritised industrial decarbonisation options. Sub-group analysis further revealed stereotype-driven patterns across gender and occupational groups, and cross-model inconsistencies compound these concerns—overall suggesting that relying on LLMs for stakeholder engagement in IAM studies risks propagating existing biases, with outputs heavily contingent on the model used.