GeneAgent: Self-verification Language Agent for Gene Set Analysis using Domain Databases

Recent work in gene set analysis has shown promising performance utilizing large language models (LLMs). Nonetheless, their results are subject to limitations common in LLMs, such as hallucinations. In response, we develop GeneAgent, a language agent for gene set analysis that self-verifies by autonomously interacting with biological databases, reducing hallucinations and enhancing accuracy. GeneAgent generates novel function names or aligns with notable enriched terms for input gene sets. Benchmarking on 1,106 gene sets from different sources, GeneAgent consistently outperforms vanilla GPT-4 by a significant margin. A detailed manual review confirms the effectiveness of the self-verification module in minimizing hallucinations and generating a more reliable explanatory analysis. We also apply GeneAgent to seven novel gene sets derived from mouse B2905 melanoma cell lines, with expert evaluations showing that GeneAgent offers valuable insights into gene functions and expediting knowledge discovery.