Papers › Dynamics of Local Elasticity During Training of Neural Nets
Dynamics of Local Elasticity During Training of Neural Nets
Soham Dan, Anirbit Mukherjee, Avirup Das, Phanideep Gampa
In the recent past, a property of neural training trajectories in weight-space had been isolated, that of "local elasticity" (denoted as Sᵣₑₗ). Local elasticity attempts to quantify the propagation of the influence of a sampled data point on the prediction at another data. In this work, we embark on a comprehensive study of the existing notion of Sᵣₑₗ and also propose a new definition that addresses the limitations that we point out for the original definition in the classification setting. On various state-of-the-art neural network training on SVHN, CIFAR-10 and CIFAR-100 we demonstrate how our new proposal of Sᵣₑₗ, as opposed to the original definition, much more sharply detects the property of the weight updates preferring to make prediction changes within the same class as the sampled data. In neural regression experiments we demonstrate that the original Sᵣₑₗ reveals a 2-phase behavior -- that the training proceeds via an initial elastic phase when Sᵣₑₗ changes rapidly and an eventual inelastic phase when Sᵣₑₗ remains large. We show that some of these properties can be analytically reproduced in various instances of doing regression via gradient flows on model predictor classes.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
No leaderboard rows for this paper in the archive.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections