Papers › Fast Differentiable Clipping-Aware Normalization and Rescaling
Fast Differentiable Clipping-Aware Normalization and Rescaling
Jonas Rauber, Matthias Bethge
Rescaling a vector δ⃗ ∈ℝⁿ to a desired length is a common operation in many areas such as data science and machine learning. When the rescaled perturbation ηδ⃗ is added to a starting point x⃗ ∈D (where D is the data domain, e.g. D = [0, 1]ⁿ), the resulting vector v⃗ = x⃗ + ηδ⃗ will in general not be in D. To enforce that the perturbed vector v is in D, the values of v⃗ can be clipped to D. This subsequent element-wise clipping to the data domain does however reduce the effective perturbation size and thus interferes with the rescaling of δ⃗. The optimal rescaling η to obtain a perturbation with the desired norm after the clipping can be iteratively approximated using a binary search. However, such an iterative approach is slow and non-differentiable. Here we show that the optimal rescaling can be found analytically using a fast and differentiable algorithm. Our algorithm works for any p-norm and can be used to train neural networks on inputs with normalized perturbations. We provide native implementations for PyTorch, TensorFlow, JAX, and NumPy based on EagerPy.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Results from the paper archive 2025-07-28
No leaderboard rows for this paper in the archive.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections