Search Papers | Poster Sessions | All Posters
Poster B125 in Poster Session B - Thursday, August 8, 2024, 1:30 – 3:30 pm, Johnson Ice Rink
Decoding of Hierarchical Inference in the Human Brain during Speech Processing with Large Language Models
Joséphine Raugel1 (), Valentin Wyart1, Jean-Rémi King2; 1Ecole normale supérieure, Laboratoire de Neurosciences Cognitives et Computationnelles, Institut National de la Santé et de la Recherche Médicale, 2Ecole normale supérieure, Laboratoire des Systèmes Perceptifs, Centre National de la Recherche Scientifique
Many theories of language in the brain rely on the notion of predictions. Yet, little is known about how linguistic predictions effectively change the representations of language in the brain. Here, we investigate how two levels of representations in the language hierarchy vary with predictability: words and phonemes. For this, we rely on Large Language Models (LLMs) trained to predict incoming words and phonemes, and estimate the posterior probability of these features as speech unfolds. We then evaluate whether predictability impacts the representations of words and phonemes decoded from the MEG responses of 27 participants listening to two hours of stories. Our results show that both words and phonemes are best decoded from the brain if they are unexpected from a given context. This finding constrains the computational architecture underlying natural speech comprehension.
Keywords: natural language processing inference magnetoencephalography language model