Papers › Nonclosedness of Sets of Neural Networks in Sobolev Spaces
Nonclosedness of Sets of Neural Networks in Sobolev Spaces
Scott Mahan, Emily King, Alex Cloninger
We examine the closedness of sets of realized neural networks of a fixed architecture in Sobolev spaces. For an exactly m-times differentiable activation function ρ, we construct a sequence of neural networks (Φₙ)_(n ∈ℕ) whose realizations converge in order-(m-1) Sobolev norm to a function that cannot be realized exactly by a neural network. Thus, sets of realized neural networks are not closed in order-(m-1) Sobolev spaces W^(m-1,p) for p ∈[1,∞]. We further show that these sets are not closed in W^(m,p) under slightly stronger conditions on the m-th derivative of ρ. For a real analytic activation function, we show that sets of realized neural networks are not closed in W^(k,p) for any k ∈ℕ. The nonclosedness allows for approximation of non-network target functions with unbounded parameter growth. We partially characterize the rate of parameter growth for most activation functions by showing that a specific sequence of realized neural networks can approximate the activation function's derivative with weights increasing inversely proportional to the Lᵖ approximation error. Finally, we present experimental results showing that networks are capable of closely approximating non-network target functions with increasing parameters via training.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Results from the paper archive 2025-07-28
No leaderboard rows for this paper in the archive.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections