Search Papers | Poster Sessions | All Posters
Poster A25 in Poster Session A - Tuesday, August 6, 2024, 4:15 – 6:15 pm, Johnson Ice Rink
Learning Dynamics and Geometry in Recurrent Neural Controllers
Ann Huang1 (), Satpreet Singh1, Kanaka Rajan1; 1Harvard University
Understanding how recurrent neural networks (RNNs) learn to perform complex tasks through interaction with an environment, i.e. as agents or controllers, is important for both artificial intelligence and neuroscience. A lot of previous work has analyzed RNNs trained using supervised learning, and relatively less attention has been paid to reinforcement learning (RL) in the context of recurrent architectures and to their learning dynamics. Here, we take a step towards addressing this gap by thoroughly analyzing the learning dynamics of RNN-based artificial agents trained by reinforcement to solve a classic nonlinear continuous control problem – the Inverted Pendulum. Our framework provides key intuitions on the evolution of the control policy, neural dynamics, representational geometry, and memory in RNN-based agents.
Keywords: recurrent neural network deep reinforcement learning learning dynamics dynamical systems