Papers › Convergence Analysis of the Dynamics of a Special Kind of Two-Layered Neural Networks...
Convergence Analysis of the Dynamics of a Special Kind of Two-Layered Neural Networks with ℓ₁ and ℓ₂ Regularization
Zhifeng Kong
In this paper, we made an extension to the convergence analysis of the dynamics of two-layered bias-free networks with one ReLU output. We took into consideration two popular regularization terms: the ℓ₁ and ℓ₂ norm of the parameter vector w, and added it to the square loss function with coefficient λ/2. We proved that when λ is small, the weight vector w converges to the optimal solution ŵ (with respect to the new loss function) with probability ≥(1-ε)(1-A_d)/2 under random initiations in a sphere centered at the origin, where ε is a small value and A_d is a constant. Numerical experiments including phase diagrams and repeated simulations verified our theory.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Results from the paper archive 2025-07-28
No leaderboard rows for this paper in the archive.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections