Search Papers | Poster Sessions | All Posters
Poster B135 in Poster Session B - Thursday, August 8, 2024, 1:30 – 3:30 pm, Johnson Ice Rink
Differential Behavioral Manifestations in Emotional versus Neutral Scene Perception within Convolutional Neural Networks
Chun-Hui Li1 (), Szu-Chi Chen1, Bo-Cheng Kuo1; 1National Taiwan University
Numerous studies have highlighted the impact of emotional information on visual perception. However, the extent to which emotional information is encoded and processed in visual representations when viewing scenes remains unclear. Here, we conducted representational similarity and variance partitioning analyses to explore the visual representations of scenes containing emotional and neutral information in convolutional neural networks. Our results indicated an increasing similarity between emotion and VGG-16 RDMs, starting from the third convolutional layer, with higher similarities observed for negative images compared to neutral ones. Moreover, variance partitioning results showed that emotion model exhibited an increasing trend in explained variance from shallow to deep layers, whereas color model revealed a converse pattern of decreasing variance. Importantly, emotion model displayed significantly higher unique explanatory power for VGG-16 RDMs when comparing negative images to neutral ones beginning at the fourth layer, suggesting emotional enhancement within visual representations. Overall, our findings demonstrated the hierarchical integration of emotional information within the visual representation underlying scene perception.
Keywords: scene perception emotion convolutional neural networks representational similarity analysis