Search Papers | Poster Sessions | All Posters
Poster B89 in Poster Session B - Thursday, August 8, 2024, 1:30 – 3:30 pm, Johnson Ice Rink
Emergence of complementary learning systems through meta-learning
Zhenglong Zhou1, Anna Schapiro1; 1University of Pennsylvania
To process information from the external world, the brain relies on a hierarchy of processing systems, which initiate in early sensory neocortical areas and converge on the hippocampus. Components of this hierarchy exhibit markedly different computational properties, with the hippocampus supporting faster plasticity and employing sparser representations. There has been extensive work on the properties of these systems, but it remains unclear how and why these systems emerged in the first place. We explore the emergence of a hierarchy of processing systems in artificial neural networks using a meta-learning approach. As networks optimize for a set of tasks, they concurrently meta-learn hyperparameters that modulate layer-wise learning rates and sparsity. We find that this meta-learning promotes superior performance, at overall higher sparsity levels. We demonstrate that key aspects of complementary learning systems emerge in the networks, with a brain-like differentiation of sparsity and learning rates across layers. Furthermore, when endowed with two pathways and trained on a task with opposing demands of individual item recognition and categorization, the models capture divergent properties between intra-hippocampal pathways. Together, these results suggest that the organization of heterogenous learning systems in the brain may arise from optimizing biological variables that govern learning rate and sparsity.
Keywords: hippocampus neocortex meta-learning