Search Papers | Poster Sessions | All Posters
Poster B165 in Poster Session B - Thursday, August 8, 2024, 1:30 – 3:30 pm, Johnson Ice Rink
Semantic action representations in the mind and brain
Diana C Dima1 (), Jody C Culham1, Yalda Mohsenzadeh1; 1Western University
Understanding others’ actions is an essential part of our everyday visual experience, yet the underlying computations are not well understood. Natural actions pose a complexity challenge, varying along many perceptual features. To address this, we annotated natural videos of everyday actions with a rich set of visual, social, and semantic features. In particular, we tested four models of action categorization defining actions at different levels of abstraction from specific (action verb) to broad (action target: an object, a person, or the self). We combined behavioral similarity judgments, EEG, and fMRI to investigate action representations in the mind and brain. Using variance partitioning, we found that the target of actions uniquely explained behavioral similarity judgments, as well as EEG patterns starting at 200 ms after video onset. EEG-fMRI fusion linked this processing stage to representations in the lateral occipitotemporal cortex. Together, our results show that actions are categorized primarily according to their target, and reveal the underlying spatiotemporal dynamics.
Keywords: action perception behavioral similarity representational similarity analysis EEG-fMRI fusion