Papers › Benefits of Additive Noise in Composing Classes with Bounded Capacity
Benefits of Additive Noise in Composing Classes with Bounded Capacity
Alireza Fathollah Pour, Hassan Ashtiani
We observe that given two (compatible) classes of functions ℱ and ℋ with small capacity as measured by their uniform covering numbers, the capacity of the composition class ℋ ∘ℱ can become prohibitively large or even unbounded. We then show that adding a small amount of Gaussian noise to the output of ℱ before composing it with ℋ can effectively control the capacity of ℋ ∘ℱ, offering a general recipe for modular design. To prove our results, we define new notions of uniform covering number of random functions with respect to the total variation and Wasserstein distances. We instantiate our results for the case of multi-layer sigmoid neural networks. Preliminary empirical results on MNIST dataset indicate that the amount of noise required to improve over existing uniform bounds can be numerically negligible (i.e., element-wise i.i.d. Gaussian noise with standard deviation 10⁻²⁴⁰). The source codes are available at https://github.com/fathollahpour/composition_noise.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Results from the paper archive 2025-07-28
No leaderboard rows for this paper in the archive.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections