Search Papers | Poster Sessions | All Posters
Poster B58 in Poster Session B - Thursday, August 8, 2024, 1:30 – 3:30 pm, Johnson Ice Rink
Multitasking leads to Generalizable Disentangled Representations in RNNs
Pantelis Vafidis1 (), Aman Bhargava1, Antonio Rangel1; 1California Institute of Technology
Abstract, or disentangled, representations are a promising mathematical framework for efficient and effective generalization in both biological and artificial systems. We investigate abstract representations in the context of multi-task classification over noisy evidence streams -- a canonical decision-making neuroscience paradigm. We derive theoretical bounds that guarantee the emergence of disentangled representations in the latent state of any optimal multi-task classifier, when the number of tasks exceeds the dimensionality of the state space. Turning to simulations, we confirm that RNNs trained on multi-task classification learn disentangled representations in the form of continuous attractors, and zero-shot generalize out-of-distribution (OOD). We demonstrate the flexibility of the abstract RNN representations across various decision boundary geometries and in tasks requiring classification confidence estimation. Closely relating to representations found in humans and animals during decision-making and spatial reasoning tasks, our framework suggests a general principle for the formation of cognitive maps that organize knowledge to enable flexible generalization in biological and artificial systems alike.
Keywords: recurrent neural networks representation learning generalization disentanglement