Search Papers | Poster Sessions | All Posters
Poster B122 in Poster Session B - Thursday, August 8, 2024, 1:30 – 3:30 pm, Johnson Ice Rink
Optimal stimulus selection for dissociating acoustic and semantic processing of natural sounds
Maria Araújo Vitoria1, Marie Plegat2, Giorgio Marinato2, Michele Esposito1, Christian Herff3, Bruno L. Giordano2, Elia Formisano1; 1Faculty of Psychology and Neuroscience, Maastricht University, 2Institut des Neurosciences de La Timone, CNRS and Université Aix-Marseille, 3School for Mental Health and Neuroscience, Maastricht University
Computational model-based analyses of behavioral and neural responses to natural sounds offer insights into the acoustic-to-semantic transformations involved in sound recognition. However, the inherent relation between low-level/intermediate features and semantic dimensions in natural stimuli complicates interpretation. Here, we present a method to identify optimal sets of natural sounds, minimizing the dependence between modeled representations at acoustic/intermediate and semantic level. We applied this approach in a behavioral experiment where participants made pairwise similarity judgments of sounds and semantic labels describing them. Our findings demonstrate that sound similarity judgements were most accurately modeled by the intermediate layers of a sound-to-event DNN (Yamnet), with minimal contribution of a semantic model (word2vec), whereas semantic similarity judgements exhibited the opposite pattern. These results highlight the effectiveness of our approach in dissociating acoustic and semantic processing of natural sounds, providing a framework for investigating further the neural computations underlying the processing of such stimuli.
Keywords: auditory semantics AI-based modeling real-world neuroscience behavior