Search Papers | Poster Sessions | All Posters
Poster B71 in Poster Session B - Thursday, August 8, 2024, 1:30 – 3:30 pm, Johnson Ice Rink
Recurrent Attentional Selection Can Explain Flexible Trading of Accuracy and Energy in Biological Vision
Eivinas Butkus1 (), Zhuofan Ying1, Peiyu Chen1, Nikolaus Kriegeskorte1; 1Columbia University
Biological vision is energetically costly. Visual attention may save energy by selecting, on the basis of a cursory initial analysis, the features and locations that deserve scrutiny. Here we investigate this idea using recurrent convolutional neural network models with graded attentional selection, implemented as multiplicative gain on features (what gain) and locations (where gain). The task for both humans and models was to determine the class (what) and location (where) of a handwritten digit among letters. Humans viewed brief presentations of such cluttered images and also rated the difficulty of each search. Models were trained with a loss encouraging high accuracy in both the what and the where task and low energy use. We found that models with attention achieved the best (Pareto-optimal) combinations of energy and accuracy. In contrast to models with no penalty on energy use, models optimized with an intermediate energy cost term consistently had a higher correlation across images between model energy use and human difficulty judgments. Finally, models that included feature-based attention (what gain) better explained human difficulty judgments. Our work demonstrates the importance of resource costs for understanding the computational mechanisms of biological vision.
Keywords: attention vision energy recurrent neural networks