Search Papers | Poster Sessions | All Posters
Poster C119 in Poster Session C - Friday, August 9, 2024, 11:15 am – 1:15 pm, Johnson Ice Rink
Network computational mechanism underlying uncertainty-driven arbitration between model-based and model-free reinforcement learning
Siyu Wang1 (), Bruno Averbeck1; 1National Institute of Mental Health
In reinforcement learning, humans and animals rely on both a deliberative model-based system which builds internal models of the environment to facilitate learning and action planning, and a habitual model-free system which reinforces actions directly from rewards. How do animals arbitrate between the two systems? Previous studies based on behavioral modeling have provided evidence that the reliability of the predictions from these two systems determine how animals arbitrate between them. However, it is unknown how such computation is implemented by populations of neurons. In this work, we investigate the computational motif in networks that underlie the arbitration between a model-based and a model-free system in reinforcement learning. We trained recurrent neural network models that can flexibly switch between model-based and model-free strategies based on the task environment. By analyzing latent network activity during the arbitration process, we show how attractor population dynamics in networks underlie the model-based vs model-free arbitration. Our results suggested a general computational motif in networks on uncertainty-driven arbitration between abstract choices.
Keywords: reinforcement learning model-based model-free population dynamics