Search Papers | Poster Sessions | All Posters
Poster A149 in Poster Session A - Tuesday, August 6, 2024, 4:15 – 6:15 pm, Johnson Ice Rink
Representational geometries supporting social action understanding in naturalistic vision
Jane Han1 (), Maria Gobbini2, James Haxby1, Samuel Nastase3; 1Dartmouth College, 2University of Bologna, Italy, 3Princeton University
To understand the actions of others, humans effortlessly extract rich, behaviorally relevant information about actions and intentions from dynamic visual input. How are our internal neural representations of observed actions structured? Here we measured brain activity using fMRI while participants viewed video clips of a wide range of naturalistic action clips. We measured the perceived similarity of the clips based on different criteria by asking participants to arrange them on a screen based on the purpose of the actions (transitive goals and type of social interaction) and visual content (person, object, scene). In a searchlight analysis, we tested the correlation of neural representational geometries derived from the arrangement tasks. Results revealed an extensive system for action representation that encompassed most visual cortex in occipital, temporal, parietal, and prefrontal cortices. Representational geometries based on sociality and transitivity judgments better predicted neural representational geometry than did geometries based on similarities of people, scenes, and objects. Transitivity better predicted neural representational geometry than sociality with notable exceptions in the precuneus, posterior superior temporal sulcus, and lateral occipitotemporal cortex bilaterally. Our findings indicate that visual representation is dominated by agentic action and is best accounted for by behavioral measures of the purpose of actions.
Keywords: action perception fMRI representational geometry hyperalignment