Search Papers | Poster Sessions | All Posters
Poster B56 in Poster Session B - Thursday, August 8, 2024, 1:30 – 3:30 pm, Johnson Ice Rink
A Mathematical Theory of Context Controlled Semantic Development
Devon Jarvis1 (), Richard Klein1, Benjamin Rosman1, Andrew Saxe2; 1University of the Witwatersrand, 2University College London
A number of phenomena which underlie human semantic cognition, and emerge during childhood, have been established. Recent work has provided a mathematical theory for the context unaware phenomena, using deep linear neural networks. Here we extend this theory to encompass aspects of cognitive control in semantic learning. This follows later in a child's development and requires the ability to ``gate'' aspects of the cognitive computation. Gating is a nonlinear process where portions of the computation are inhibited in some contexts. We use a neural network with ReLU activation to perform the gating and model three more behaviours in semantic development, namely domain specific attribute weighting, new attribute induction and conceptual reorganisation. We use a Gated Deep Linear Network to model the ReLU network, providing the full training dynamics and interpretability in its implementation of cognitive control. We find that the ReLU network uses an intricately structured latent representation which is mixed selective. Thus, we demonstrate how reusable, generalizable and mixed-selective latent representations may emerge, three properties which have previously seemed incongruent.
Keywords: Controlled Cognition Semantic Learning Gated Linear Networks Mixed Selectivity