{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cosine-normalization-using-cosine-similarity","title":"Cosine Normalization: Using Cosine Similarity Instead of Dot Product in Neural Networks","arxiv_id":"1702.05870","date":"2017-02-20","proceeding":null,"authors":["Chunjie Luo","Jianfeng Zhan","Lei Wang","Qiang Yang"],"abstract":"Traditionally, multi-layer neural networks use dot product between the output\nvector of previous layer and the incoming weight vector as the input to\nactivation function. The result of dot product is unbounded, thus increases the\nrisk of large variance. Large variance of neuron makes the model sensitive to\nthe change of input distribution, thus results in poor generalization, and\naggravates the internal covariate shift which slows down the training. To bound\ndot product and decrease the variance, we propose to use cosine similarity or\ncentered cosine similarity (Pearson Correlation Coefficient) instead of dot\nproduct in neural networks, which we call cosine normalization. We compare\ncosine normalization with batch, weight and layer normalization in\nfully-connected neural networks as well as convolutional networks on the data\nsets of MNIST, 20NEWS GROUP, CIFAR-10/100 and SVHN. Experiments show that\ncosine normalization achieves better performance than other normalization\ntechniques.","url_abs":"http://arxiv.org/abs/1702.05870v5","url_pdf":"http://arxiv.org/pdf/1702.05870v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cosine-normalization-using-cosine-similarity","repo_url":"https://github.com/iwyoo/tf_conv_cosnorm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[{"method_slug":"cosine-normalization","method_name":"Cosine Normalization"}],"datasets_introduced":[],"methods_introduced":[{"slug":"cosine-normalization","name":"Cosine Normalization","full_name":"Cosine Normalization"}],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1702.05870","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1702.05870"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/iwyoo/tf_conv_cosnorm","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":1},"by_repo_kind":{"listed":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"bcb40d866498afc6","entry":"conv2d_cosnorm","repo":"iwyoo/tf_conv_cosnorm","repo_kind":"listed","path":"conv_cosnorm.py","file_url":"https://github.com/iwyoo/tf_conv_cosnorm/blob/HEAD/conv_cosnorm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bcb40d866498afc6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}