Papers › A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-Off
A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-Off
Yaniv Blumenfeld, Dar Gilboa, Daniel Soudry
Reducing the precision of weights and activation functions in neural network training, with minimal impact on performance, is essential for the deployment of these models in resource-constrained environments. We apply mean-field techniques to networks with quantized activations in order to evaluate the degree to which quantization degrades signal propagation at initialization. We derive initialization schemes which maximize signal propagation in such networks and suggest why this is helpful for generalization. Building on these results, we obtain a closed form implicit equation for Lₘₐₓ, the maximal trainable depth (and hence model capacity), given N, the number of quantization levels in the activation function. Solving this equation numerically, we obtain asymptotically: Lₘₐₓ∝N^(1.82).
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
No leaderboard rows for this paper in the archive.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections