Search Papers | Poster Sessions | All Posters
Poster C54 in Poster Session C - Friday, August 9, 2024, 11:15 am – 1:15 pm, Johnson Ice Rink
Towards Task-Appropriate Readout Mechanisms For Physical Scene Understanding
Khaled Jedoui Al-Karkari1 (), Rahul Venkatesh1, Haoliang Wang2, Thomas O'Connell3, Yoon H. Bai3, Joshua B. Tenenbaum3, Judith E. Fan1, Kevin A. Smith3, Daniel L.K. Yamins1; 1Stanford University, 2University of California, San Diego, 3Massachusetts Institute of Technology
Establishing robust readouts is essential in the study and interpretation of both artificial intelligence models and the brain. These linking functions extract relevant information that allows us to understand model behavior, as well as brain activity patterns. However, building appropriate readouts for any given task has been challenging due to the lack of clear strategies and methods to design such functions. In this paper, we propose an approach to derive a readout model of least complexity using an idealized data representation (with all information needed to solve a task). We investigate the ability of our model to decode representations for a simple physics understanding task; object contact detection. We demonstrate that our approach provides a better qualitative signal about AI models, compared to the traditional linear readout. Our readout promises to not only improve the benchmarking of AI models, but also provides a path forward for building more powerful neural decoders and gaining insight into how different brain regions represent and reason about the physical world.
Keywords: AI Neuroscience Cognitive Benchmarking Readout