Search Papers | Poster Sessions | All Posters
Poster C11 in Poster Session C - Friday, August 9, 2024, 11:15 am – 1:15 pm, Johnson Ice Rink
Constraining vision models to predict image memorability yields significant gains in producing more brain-aligned models of the primate ventral stream
Ram Ahuja1 (), Soroush Ziaee1, Ezgi Fide1, Shayna Rosenbaum1, Kohitij Kar1; 1York University
The primate ventral visual stream that culminates in the inferior temporal (IT) cortex supports critical functions, including object recognition and visual memory. Previous work has demonstrated that artificial neural networks (ANNs) optimized for object categorization exhibit unprecedented but partial alignment with ventral stream representations. However, it remains unknown whether ANNs constrained to predict human image memorability could explain a unique part of the neural variance -- potentially bridging the remaining explanatory gap. We observed that models trained to predict image memorability predict unique variances of the IT neural responses. Interestingly, joint categorization and memorability training yielded networks that captured significantly more variance in neural responses than models trained on either objective alone. Our results suggest that incorporating diverse, functionally relevant objectives leads to ANNs more closely aligned with the primate ventral visual stream's representational geometry and functional properties.