Reinforcement learning links spontaneous cortical dopamine impulses to reward

Conrad Foo, Adrian Lozada, Johnatan Aljadeff, Yulong Li, Jing W. Wang, Paul A. Slesinger, David Kleinfeld

Research output: Contribution to journalArticlepeer-review

10 Scopus citations

Abstract

In their pioneering study on dopamine release, Romo and Schultz speculated “…that the amount of dopamine released by unmodulated spontaneous impulse activity exerts a tonic, permissive influence on neuronal processes more actively engaged in preparation of self-initiated movements….”1 Motivated by the suggestion of “spontaneous impulses,” as well as by the “ramp up” of dopaminergic neuronal activity that occurs when rodents navigate to a reward,2–5 we asked two questions. First, are there spontaneous impulses of dopamine that are released in cortex? Using cell-based optical sensors of extrasynaptic dopamine, [DA]ex,6 we found that spontaneous dopamine impulses in cortex of naive mice occur at a rate of ∼0.01 per second. Next, can mice be trained to change the amplitude and/or timing of dopamine events triggered by internal brain dynamics, much as they can change the amplitude and timing of dopamine impulses based on an external cue?7–9 Using a reinforcement learning paradigm based solely on rewards that were gated by feedback from real-time measurements of [DA]ex, we found that mice can volitionally modulate their spontaneous [DA]ex. In particular, by only the second session of daily, hour-long training, mice increased the rate of impulses of [DA]ex, increased the amplitude of the impulses, and increased their tonic level of [DA]ex for a reward. Critically, mice learned to reliably elicit [DA]ex impulses prior to receiving a reward. These effects reversed when the reward was removed. We posit that spontaneous dopamine impulses may serve as a salient cognitive event in behavioral planning.

Original languageEnglish
Pages (from-to)4111-4119.e4
JournalCurrent Biology
Volume31
Issue number18
DOIs
StatePublished - 27 Sep 2021

Keywords

  • biophysical modeling
  • brain machine interface
  • classical conditioning
  • feedback
  • foraging
  • neuromodulation
  • stochastic dynamics
  • two-photon microscopy

Fingerprint

Dive into the research topics of 'Reinforcement learning links spontaneous cortical dopamine impulses to reward'. Together they form a unique fingerprint.

Cite this