Search Papers | Poster Sessions | All Posters
Poster C79 in Poster Session C - Friday, August 9, 2024, 11:15 am – 1:15 pm, Johnson Ice Rink
Improving Neural Decoding by Integrating Information Over Time
Zhuoyang (Gio) Li1, Kristijan Armeni1, Christopher J. Honey1; 1Johns Hopkins University
Deep language models (DLMs) provide a powerful basis for building encoding models and decoding models of neural responses. However, DLMs are usually only used to predict a single moment in the neural signal. We reasoned that decoding performance might be improved by combining information across time, and leveraging the temporal dependencies of neural activity. To test this, we analyzed word-level decoding from electrocorticography (ECoG) recordings of 9 participants who listened to a 7-minute narrative. We found that DLM-based encoding models could predict neural responses seconds before and after a word onset, and that the predictions did not generalize across time intervals around word onset. Moreover, we were able to boost decoder performance by integrating information across distinct time intervals. Thus, human brains represent diverse word-related information for hundreds of milliseconds before and after word onset, and ensembling information over time is a promising approach for naturalistic neural decoding.
Keywords: ECoG language models neural decoding temporal integration