Analyzing the Advantages of Utilizing State Representations in a Probabilistic Reversal Learning Task

Abstract: Cognitive flexibility is the ability to adaptively change behaviors in the face of dynamically changing circumstances. To explore the neural basis and computational account of this ability, a probabilistic reversal learning task was employed as the experimental paradigm. Recent studies suggest that a subject may utilize not only a reward history but also a “state representation” of a task to successfully solve one. However, the specific advantages or impact of state representations in task solving are still not fully understood. In this study, we investigated this matter by computer simulations, in which we used two types of reinforcement learning models, a model with state representations and one without. As a result of the simulations, we found that state representations make a learning agent robust against an increasingly difficult task, especially when the number of sampling time in each state is reduced. Based on the results, we propose a hypothesis for the acquisition process of state representations and discuss the experimental design to test it.

Keywords: Cognitive flexibility; Probabilistic reversal learning task; Reinforcement learning model; State representations