Addressing the credit assignment problem in treatment outcome prediction using temporal difference learning

Sahar Harati, Andrea Crowell, Helen Mayberg, Shamim Nemati

Research output: Contribution to journalConference articlepeer-review

2 Scopus citations


Mental health patients often undergo a variety of treatments before finding an effective one. Improved prediction of treatment response can shorten the duration of trials. A key challenge of applying predictive modeling to this problem is that often the effectiveness of a treatment regimen remains unknown for several weeks, and therefore immediate feedback signals may not be available for supervised learning. Here we propose a Machine Learning approach to extracting audio-visual features from weekly video interview recordings for predicting the likely outcome of Deep Brain Stimulation (DBS) treatment several weeks in advance. In the absence of immediate treatment-response feedback, we utilize a joint state-estimation and temporal difference learning approach to model both the trajectory of a patient's response and the delayed nature of feedbacks. Our results based on longitudinal recordings from 12 patients with depression show that the learned state values are predictive of the long-Term success of DBS treatments. We achieve an area under the receiver operating characteristic curve of 0.88, beating all baseline methods.

Original languageEnglish
Pages (from-to)43-54
Number of pages12
JournalPacific Symposium on Biocomputing
Issue number2020
StatePublished - 2020
Externally publishedYes
Event25th Pacific Symposium on Biocomputing, PSB 2020 - Big Island, United States
Duration: 3 Jan 20207 Jan 2020


  • Depression.
  • Machine Learning
  • Temporal Difference Learning


Dive into the research topics of 'Addressing the credit assignment problem in treatment outcome prediction using temporal difference learning'. Together they form a unique fingerprint.

Cite this