Search Papers | Poster Sessions | All Posters
Poster C116 in Poster Session C - Friday, August 9, 2024, 11:15 am – 1:15 pm, Johnson Ice Rink
Towards time-scale invariant reinforcement learning
Md Rysul Kabir1, James Mochizuki-Freeman1, Zoran Tiganj1 (); 1Indiana University Bloomington
The ability to estimate temporal relationships is critical for both animals and artificial agents. Cognitive science and neuroscience studies provide remarkable insights into behavioral and neural aspects of interval timing. In particular, scale-invariance observed in behavior and supported by neural data is one of the key principles that goes beyond interval timing and governs animal perception. Furthermore, once they learn a task, humans and other animals can rapidly adapt to temporally rescaled versions at a wide range of scales. We show that using a scale-invariant cognitive model of working memory combined with convolutional and max-pool layers gives rise to reinforcement learning (RL) agents that are invariant to temporal rescaling in the environment. We illustrate this using a simple interval bisection task and show that this property is specific to scale-invariant memory and not observed in commonly used recurrent networks.
Keywords: Scale-invariance Deep RL Interval timing