Search Papers | Poster Sessions | All Posters
Poster A24 in Poster Session A - Tuesday, August 6, 2024, 4:15 – 6:15 pm, Johnson Ice Rink
Understanding and Optimizing Temporal Credit Assignment in Biological and Artificial Neural Networks using Dynamical Systems Theory
Rainer Engelken1 (), Larry Abbott; 1Columbia University
Finding temporal associations across long timescales in cognitively demanding tasks, such as delayed match to sample and parametric working memory, is challenging for both biological and artificial neural networks. Gradient-based training of recurrent neural circuit models for temporal tasks with long time horizons presents challenges that potentially lead to vanishing or exploding gradients. We leverage dynamical systems theory to understand the learning dynamics and solution space of such temporal credit assignment problems in spiking and firing rate networks. Specifically, we connect this issue to the Lyapunov exponents of the forward dynamics, describing how perturbations grow or shrink during forward passes. We propose "gradient flossing", a method to address gradient instability in recurrent spiking and firing rate networks by controlling the Lyapunov exponents of the forward dynamics throughout learning. We regularize Lyapunov exponents towards zero, ensuring that the corresponding directions in tangent space grow or shrink only slowly to facilitate more robust propagation of learning signals over long time horizons. This approach improves RNN stability and training success in temporal cognitive tasks by regulating the norm and dimensionality of the gradient signal in backpropagation through the dynamic adjustment of Lyapunov exponents.
Keywords: Temporal Credit Assigment Learning Dynamics Backpropagation Through Time Exploding/Vanishing Gradients