Search Papers | Poster Sessions | All Posters
Poster A15 in Poster Session A - Tuesday, August 6, 2024, 4:15 – 6:15 pm, Johnson Ice Rink
Distinct policy identification via model-based belief update
Panos Alefantis1, Zhe Li3, Noushin Quazi2, Dora E Angelaki1, Xaq Pitkow2; 1New York University, 2Carnegie Mellon, 3Baylor Collage of Medicine
Both artificial and biological agents face the challenge of interacting with the world based on incomplete and noisy observations. In general, the agent relies on a latent representation that integrates past experience, and makes decisions based on it. When the agent has an internal model of the environment, its latent representation can be a distribution about the unknown state, i.e., belief. A major challenge in using this framework is that the model’s belief dynamics are difficult to derive analytically, and this approach is not scalable to complex environments. We developed an algorithm that automates this process, requiring only a simulator of the environment dynamics that the agent assumes. Our approach uses this simulator to estimate dynamics of the internal model beliefs. We applied our algorithm to explain behavioral data of a monkey foraging experiment, where the monkey navigates between three food boxes and decides when to open the box based on noisy color cues. We found the beliefs span on a low dimensional manifold in the belief space, and they are well organized by behaviorally relevant variables. We also developed an algorithm to identify distinct modes of behavior, accommodating potentially non-stationary policies. The algorithm was on simulated behavior data, and show an initial application to real behavior, revealing changes in strategy over time. In summary, our tools can infer the latent representation of model-based agents and the policy defined over this representation, which will enable a search for neural evidence of possible internal model properties that are not directly measurable.