Search Papers | Poster Sessions | All Posters
Poster B80 in Poster Session B - Thursday, August 8, 2024, 1:30 – 3:30 pm, Johnson Ice Rink
Modeling the human brain: investigating stimulus-response transformations in complex, time-continuous environments using deep neural networks
Sabine Haberland1 (), Hannes Ruge1, Holger Frimmel1; 1TUD Dresden University of Technology
Studying how humans transform complex, high-dimensional stimuli into appropriate behavior within time-continuous environments has been challenging due to the prevalent use of trial-based study designs. Deep Q-Networks (DQNs) have emerged as valuable tools for modeling stimulus-response (S-R) transformations in such environments. Here, we showed that a DQN-based encoding model approach can be used to predict neural activations and human behavior using features generated by a DQN. Therefore, we collected motor responses and fMRI data from human subjects (N=23) while playing arcade games. We hypothesized that advancements in machine learning can be leveraged to improve prediction accuracy. We compared the prediction accuracy of features generated by two recently developed DQNs and a third baseline DQN, each differing in network architecture and training procedures. We present preliminary evidence that all three DQNs predict behavior and fMRI activations significantly above chance at a fine-grained temporal scale. Features generated by the most advanced model achieved the best results. We found a hierarchical correspondence between the layers of DQNs and stages of human visuo-motor processing. These findings suggest that improved DQNs serve as suitable tools for modeling S-R transformations in a time-continuous manner.
Keywords: Encoding Model Deep Neural Network Neuroimaging Arcade Games