Search Papers | Poster Sessions | All Posters
Poster A14 in Poster Session A - Tuesday, August 6, 2024, 4:15 – 6:15 pm, Johnson Ice Rink
Human Behavior is Best Predicted by a Novel Successor Representation Learning Rule
Ari Kahn1, Dani Bassett2, Nathaniel Daw1; 1Princeton University, 2University of Pennsylvania
Human decision making depends on learning and using models that capture the statistical structure of the world, including the long-run expected outcomes of our actions. One prominent approach to abstracting such long-run outcomes is the successor representation (SR), which represents a mapping between current and future states, and has been implicated in both behavioral and neural data. Although much behavioral and neural evidence suggest that people and animals use such a representation, it remains unknown how they learn it. Bootstrapping methods (SR-TD(0)) have been ubiquitously proposed, but bootstrapping a vector-valued function in large state spaces appears biologically implausible. Here we propose an alternative learning rule, termed SR-Trace, which approximates SR-TD(1) using a simpler scalar update process. We examined the behavior of both on a probabilistic graph learning task, and found that trial-by-trial response times were better predicted by the more plausibly realizable SR-Trace model, suggesting that humans may rely on this biologically plausible SR learning rule in graph learning tasks.
Keywords: reinforcement learning graph learning successor representation predictive models