Search Papers | Poster Sessions | All Posters
Poster B82 in Poster Session B - Thursday, August 8, 2024, 1:30 – 3:30 pm, Johnson Ice Rink
Visual Feature-Based Brain Decoding Yields Weight Maps Better Aligned with Scene Understanding than Classification
Chenqian Le1, Nikasadat Emami1, Xujin Chris Liu1, Xupeng Chen1, Yao Wang1; 1New York University
We introduce a brain decoding method for analyzing functional responses to visual perception using the Natural Scenes Dataset (NSD), where we use visual features of images from deep neural networks as a decoding target. Our method gives consistent results across various feature extraction methods and subjects. Using the resulting weight map in a follow-up classification task, our method achieves similar classification accuracy as a directly trained classifier yet offers broader applicability since no classification labels are needed. We show that our resulting weight maps are more closely aligned with the underlying task of human subjects compared to weight maps derived from classification-based decoding. The flexibility makes our method suitable for diverse decoding-style analysis with complex stimuli, where manual labeling might bias the results.
Keywords: Deep neural network Brain visual decoding fMRI