Search Papers | Poster Sessions | All Posters
Poster B51 in Poster Session B - Thursday, August 8, 2024, 1:30 – 3:30 pm, Johnson Ice Rink
Why Do Animals Need Shaping? A Theory of Task Composition and Curriculum Learning
Jin Lee1 (), Stefano Sarao Mannelli1, Andrew Saxe1; 1UCL
Diverse studies in systems neuroscience begin with extended periods of training known as 'shaping' procedures. These involve progressively training on components of more complex tasks, and can make the difference between learning a task quickly, slowly or not at all. Despite the importance of shaping to the acquisition of complex tasks, there is as yet no theory that can help guide the design of shaping procedures, or more fundamentally, provide insight into its key role in learning. In this light, we propose and analyse a model of deep policy gradient learning on compositional reinforcement learning (RL) tasks. Using the tools of statistical physics, we solve exactly the learning dynamics and characterise different learning strategies including primitives pre-training, in which task primitives are studied individually before learning compositional tasks. We find a complex interplay between task complexity and the efficacy of shaping strategies. Overall, our theory provides an analytical understanding of the benefits of shaping in a class of compositional tasks and a quantitative account of how training protocols can disclose useful task primitives, ultimately yielding faster and more robust learning.
Keywords: learning theory compositionality curriculum learning