Search Papers | Poster Sessions | All Posters
Poster C60 in Poster Session C - Friday, August 9, 2024, 11:15 am – 1:15 pm, Johnson Ice Rink
Larger Language Models Better Predict Neural Activity During Natural Language Processing
Haocheng Wang1, Zhuoqiao Hong1, Zaid Zada2, Harshvardhan Gazula3, Bobbi Aubrey1, Werner Doyle4, Sasha Devore4, Patricia Dugan4, Daniel Friedman4, Orrin Devinsky4, Adeen Flinker4, Uri Hasson1,2, Samuel Nastase1, Ariel Goldstein5; 1Princeton Neuroscience Institute, 2Department of Psychology, Princeton University, 3McGovern Institute for Brain Research, MIT, 4New York University Grossman School of Medicine, 5Department of Cognitive and Brain Sciences, The Hebrew University Jerusalem
Recent research has used large language models (LLMs) to study the neural basis of naturalistic language processing in the human brain. LLMs have grown in complexity, leading to improved language processing capabilities. Here, we utilized several families of transformer-based LLMs to investigate the relationship between model size and their ability to capture linguistic information in the brain. Crucially, a subset of LLMs were trained on a fixed training set, enabling us to dissociate model size from architecture and training set size. We used electrocorticography (ECoG) to measure neural activity in epilepsy patients while they listened to a 30-minute naturalistic audio story. We fit electrode-wise encoding models using contextual embeddings extracted from each hidden layer of the LLMs to predict word-level neural signals. In line with prior work, we found that larger LLMs better capture the structure of natural language and better predict neural activity. We also found a log-linear relationship where the encoding performance peaks in relatively earlier layers as model size increases.
Keywords: language comprehension LLM ECoG speech