Reinforcement learning links spontaneous dopamine transients to reward

In their pioneering study on dopamine release, Romo and Schultz speculated "... that the amount of dopamine released by unmodulated spontaneous impulse activity exerts a tonic, permissive influence on neuronal processes more actively engaged in preparation of self-initiated movements, ...". Motivated by the suggestion of "spontaneous impulses", we asked two questions. First, are there spontaneous impulses of dopamine that are released in cortex? This possibility is further motivated by the "ramp up" of dopaminergic neuronal activity that occurs when rodents navigate to a reward. Using cell-based optical sensors of extrasynaptic dopamine, [DA]_ex, we found that spontaneous dopamine impulses in cortex of naive mice occur at a rate of ~ 0.01 per second. Next, can mice be trained to change the amplitude and/or timing of dopamine events triggered by internal brain dynamics, much as they can change the amplitude and timing of dopamine impulses based on an external cue? Using a reinforcement learning paradigm based solely on rewards that were gated by feedback from real-time measurements of [DA]_ex, we found that mice can volitionally modulate their spontaneous [DA]_ex. In particular, by only the second session of daily, hour-long training, mice increased the rate of impulses of [DA]_ex, increased the amplitude of the impulses, and increased their tonic level of [DA]_ex for a reward. Critically, mice learned to reliably elicit [DA]_ex impulses prior to receiving a reward. These effects reversed when the reward was removed. We posit that spontaneous dopamine impulses may serve as a salient cognitive event in behavioral planning.