TY - GEN
T1 - Application of the actor-critic architecture to functional electrical stimulation control of a human arm
AU - Thomas, Philip
AU - Branicky, Michael
AU - Van Den Bogert, Antonie
AU - Jagodnik, Kathleen
PY - 2009
Y1 - 2009
N2 - Clinical tests have shown that the dynamics of a human arm, controlled using Functional Electrical Stimulation (FES), can vary significantly between and during trials. In this paper, we study the application of the actor-critic architecture, with neural networks for the both the actor and the critic, as a controller that can adapt to these changing dynamics of a human arm. Development and tests were done in simulation using a planar arm model and Hill-based muscle dynamics. We begin by training it using a Proportional Derivative (PD) controller as a supervisor. We then make clinically relevant changes to the dynamics of the arm and test the actor-critic's ability to adapt without supervision in a reasonable number of episodes. Finally, we devise methods for achieving both rapid learning and long-term stability.
AB - Clinical tests have shown that the dynamics of a human arm, controlled using Functional Electrical Stimulation (FES), can vary significantly between and during trials. In this paper, we study the application of the actor-critic architecture, with neural networks for the both the actor and the critic, as a controller that can adapt to these changing dynamics of a human arm. Development and tests were done in simulation using a planar arm model and Hill-based muscle dynamics. We begin by training it using a Proportional Derivative (PD) controller as a supervisor. We then make clinically relevant changes to the dynamics of the arm and test the actor-critic's ability to adapt without supervision in a reasonable number of episodes. Finally, we devise methods for achieving both rapid learning and long-term stability.
UR - http://www.scopus.com/inward/record.url?scp=74949094130&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:74949094130
SN - 9781577354239
T3 - Proceedings of the 21st Innovative Applications of Artificial Intelligence Conference, IAAI-09
SP - 165
EP - 172
BT - Proceedings of the 21st Innovative Applications of Artificial Intelligence Conference, IAAI-09
T2 - 21st Innovative Applications of Artificial Intelligence Conference, IAAI-09
Y2 - 14 July 2009 through 16 July 2009
ER -