Search Papers | Poster Sessions | All Posters
Contributed Talks III
Talk Session: Thursday, August 8, 2024, 4:00 – 5:00 pm, Kresge Hall
4:00 pm
Intracranial recordings reveal neural encoding of attention-modulated reinforcement learning in humans
Christina Maher1,2 (), Salman Qasim1,2, Lizbeth Nuñez Martinez1,4,5,6, Ignacio Saez1,4,5,6, Angela Radulescu1,3,4; 1Icahn School of Medicine at Mount Sinai, 2Friedman Brain Institute, 3Department of Psychiatry, 4Department of Neuroscience, 5Department of Neurosurgery, 6Department of Neurology
Reinforcement learning (RL) is tractable in multidimensional environments when agents maintain efficient state representations, or mental models of relevant information. Attention supports state representations in service of RL by constraining learning to relevant dimensions. However, the physiological processes supporting value updating and attentional control are unknown. To investigate the neural mechanism supporting these processes we relate attention-modulated RL models to neuronal activity recorded directly from the prefrontal cortex of neurosurgical patients playing a multidimensional decision-making task. These models revealed that participants deploy selective attention during RL. Model-estimated expected value of the chosen stimulus correlated with neuronal activity in the orbitofrontal (OFC) and lateral prefrontal cortex (LFPC), though value signals in the LPFC were additionally biased by model-estimated attention. In sum, these results provide mechanistic insight into the neuronal implementation of the computations involved in attention-modulated RL.
4:12 pm
Control adaptation through Selective Suppression of Multidimensional Distractors
Davide Gheza1 (), Thea R. Zalabak1, Wouter Kool1; 1Washington University in St Louis
Humans manage multiple conflicting sources of information. However, models of cognitive control assume one source of interference and do not explain how we handle multiple distractors. In our multi-dimensional task-set interference paradigm, individuals manage distraction from three independent dimensions. Experiment 1 suggests that people use prior conflict from each dimension to selectively modulate their gain. A neural network, measuring multivariate conflict as energy within each dimension’s pathway, captures this effect. Representational similarity analyses of human EEG (Experiment 2) confirmed the selective suppression of distractor representations. These results reveal the striking human ability to simultaneously adjust attention to multiple sources of information. Model predictions converge with recent work suggesting that neural conflict signals emerge from the integration of diverse task variables in medial prefrontal cortex.
4:24 pm
Inverse reinforcement learning captures value representations in the reward circuit in a real-time driving task: a preliminary study
Sang Ho Lee1 (), Min-hwan Oh1, Woo-Young Ahn1; 1Seoul National University
A challenge in using naturalistic tasks is to describe complex data beyond simple summary of behaviors. Lee et al. (2024) showed that an inverse reinforcement learning (IRL) algorithm combined with deep neural networks is a practical framework for modeling real-time behaviors in a naturalistic task. However, it remains unknown whether the reward function inferred by IRL reflects value representations in the reward circuit. In this preliminary study (N=10), we investigate the neural correlates of the reward inferred by IRL. Human participants were scanned using fMRI while performing a real-time driving task (i.e., highway task). We show that the trajectory of IRL reward during the task strongly correlates with the trajectory of BOLD signals in the reward circuit including the prefrontal cortex, the striatum, and the insula. The results demonstrate the validity of the IRL as a modeling framework that explains both behaviors and the brain activity in a real-time task.
4:36 pm
Discrete boundaries between neural populations in recurrent neural networks
Jacob Tanner1, Sina Mansour L.2, Ludovico Coletta3, Alessandro Gozzi4, Richard Betzel1; 1Indiana University, 2National University of Singapore, 3Fondazione Bruno Kessler, 4Istituto Italiano di Tecnologia
Recent theoretical and experimental work in neuroscience has focused on the representational and dynamical character of neural manifolds [e.g. (Ebitz & Hayden, 2021; Saxena & Cunningham, 2019; Mante, Sussillo, Shenoy, & Newsome, 2013; Cunningham & Yu, 2014)]. These neural manifolds are subspaces in neural activity space wherein many neurons coactivate. Importantly, neural populations studied under this “neural manifold hypothesis” are not cleanly divided into separate neural populations. Instead, many neurons contribute to most manifolds in some way or another. Here, we leveraged RNNs as a model system to study the character of discrete neural populations. We used a community detection method from network science to produce a partition that separates neurons into distinct populations. These partitions allowed us to ask the following question: do these discrete boundaries between neural populations matter to the system? We found evidence that these boundaries do matter to the system. First, we found that these boundaries neatly divide the representational content and role of neurons. Next, we found that these boundaries can be directly inferred from features of the weight matrix and we corroborated this result with structural and functional imaging data from mice and humans. Finally, we found that the dynamics of these RNNs respected the boundaries of neurons into distinct populations.
4:48 pm
Neurocomputational mechanisms of motivational influences on mental effort
Debbie Yee1 (), Mahalia Prater Fahey2, Xiamin Leng3, Ziwei Cheng4, Maisy Tarlow5, Joonhwa Kim6, Kaitlyn Mundy7, Samuel Nevins8, Amitai Shenhav9; 1Brown University
Human motivation is fundamentally shaped by one’s expectations of their outcomes (e.g., reward, punishment), as well as the type of effort required to attain these outcomes (e.g., attention vs. caution). In our fMRI study (n=100), we observed a dissociation between how rewards promoted increased attentional control (drift rate) vs. how penalties promoted increased caution (decision threshold). We found that a priori brain regions were associated with faster RT or increased accuracy. Model-based fMRI analyses revealed rostral vs. caudal dorsal anterior cingulate cortex regions are associated with drift rate and threshold, respectively. Together, these data reveal that distinct dACC regions underlie how motivational incentives can drive attention-related and caution-related strategies for the adaptive allocation of cognitive control.