Methods › General › Normalization › Cosine Normalization
Cosine Normalization
Introduced by Chunjie Luo et al. in Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Multi-layer neural networks traditionally use dot products between the output vector of previous layer and the incoming weight vector as the input to activation function. The result of dot product is unbounded. To bound dot product and decrease the variance, Cosine Normalization uses cosine similarity or centered cosine similarity (Pearson Correlation Coefficient) instead of dot products in neural networks.
Using cosine normalization, the output of a hidden unit is computed by:
o = f(netₙₒᵣₘ)= f(cosθ) = f((w⃗ ·x⃗)/(|w⃗| |x⃗|))
where netₙₒᵣₘ is the normalized pre-activation, w⃗ is the incoming weight vector and x⃗ is the input vector, (·) indicates dot product, f is nonlinear activation function. Cosine normalization bounds the pre-activation between -1 and 1.
Papers archive 2025-07-28
2 shown of 2, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
Class-incremental Learning with Rectified Feature-Graph Preservation 15 Dec 2020 · 1 repository · arXiv:2012.08129
-
Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks 20 Feb 2017 · 1 repository · arXiv:1702.05870Syntology ran 0 of 1 samples · 1 unverified
Tasks archive 2025-07-28
3 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections