{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/weight-normalization-a-simple","title":"Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks","arxiv_id":"1602.07868","date":"2016-02-25","proceeding":"NeurIPS 2016 12","authors":["Tim Salimans","Diederik P. Kingma"],"abstract":"We present weight normalization: a reparameterization of the weight vectors\nin a neural network that decouples the length of those weight vectors from\ntheir direction. By reparameterizing the weights in this way we improve the\nconditioning of the optimization problem and we speed up convergence of\nstochastic gradient descent. Our reparameterization is inspired by batch\nnormalization but does not introduce any dependencies between the examples in a\nminibatch. This means that our method can also be applied successfully to\nrecurrent models such as LSTMs and to noise-sensitive applications such as deep\nreinforcement learning or generative models, for which batch normalization is\nless well suited. Although our method is much simpler, it still provides much\nof the speed-up of full batch normalization. In addition, the computational\noverhead of our method is lower, permitting more optimization steps to be taken\nin the same amount of time. We demonstrate the usefulness of our method on\napplications in supervised image recognition, generative modelling, and deep\nreinforcement learning.","url_abs":"http://arxiv.org/abs/1602.07868v3","url_pdf":"http://arxiv.org/pdf/1602.07868v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"weight-normalization-a-simple","repo_url":"https://github.com/TimSalimans/weight_norm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"weight-normalization-a-simple","repo_url":"https://github.com/0h-n0/weight_normalization","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"weight-normalization-a-simple","repo_url":"https://github.com/UrosOgrizovic/SimpleGoogleQuickdraw","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"weight-normalization-a-simple","repo_url":"https://github.com/astanway/gated-conv-nets","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"weight-normalization-a-simple","repo_url":"https://github.com/krasserm/wdsr","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"weight-normalization-a-simple","repo_url":"https://github.com/openai/weightnorm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"weight-normalization-a-simple","repo_url":"https://github.com/seanpmorgan/tf-weightnorm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"weight-normalization-a-simple","repo_url":"https://github.com/xiangyu-liu/Computer-Network","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"weight-normalization-a-simple","repo_url":"https://github.com/zoli333/Weight-Normalization","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"deep-reinforcement-learning","task_name":"Deep Reinforcement Learning"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"speed","method_name":"SPEED"},{"method_slug":"weight-normalization","method_name":"Weight Normalization"}],"datasets_introduced":[],"methods_introduced":[{"slug":"weight-normalization","name":"Weight Normalization","full_name":"Weight Normalization"}],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1602.07868","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1602.07868"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/seanpmorgan/tf-weightnorm","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/openai/weightnorm","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/TimSalimans/weight_norm","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/UrosOgrizovic/SimpleGoogleQuickdraw","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zoli333/Weight-Normalization","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xiangyu-liu/Computer-Network","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/astanway/gated-conv-nets","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/krasserm/wdsr","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/0h-n0/weight_normalization","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":1,"unverified":15},"by_repo_kind":{"listed":{"samples":16,"ran":1,"repositories":4}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"3ba6aec873833b39","entry":"int_shape","repo":"openai/weightnorm","repo_kind":"listed","path":"tensorflow/nn.py","file_url":"https://github.com/openai/weightnorm/blob/HEAD/tensorflow/nn.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3ba6aec873833b39"}},{"code_sha256_prefix":"5be6863211220d86","entry":"data_dependent_weight_norm_g_init","repo":"zoli333/Weight-Normalization","repo_kind":"listed","path":"model.py","file_url":"https://github.com/zoli333/Weight-Normalization/blob/HEAD/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5be6863211220d86"}},{"code_sha256_prefix":"e0f9b50298061760","entry":"evaluate","repo":"krasserm/wdsr","repo_kind":"listed","path":"model/common.py","file_url":"https://github.com/krasserm/wdsr/blob/HEAD/model/common.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e0f9b50298061760"}},{"code_sha256_prefix":"c84c1ed8059aa2ed","entry":"get_image_from_images","repo":"UrosOgrizovic/SimpleGoogleQuickdraw","repo_kind":"listed","path":"image_operations.py","file_url":"https://github.com/UrosOgrizovic/SimpleGoogleQuickdraw/blob/HEAD/image_operations.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c84c1ed8059aa2ed"}},{"code_sha256_prefix":"0651dc674036b1ce","entry":"get_model","repo":"UrosOgrizovic/SimpleGoogleQuickdraw","repo_kind":"listed","path":"models/SVM/SVM.py","file_url":"https://github.com/UrosOgrizovic/SimpleGoogleQuickdraw/blob/HEAD/models/SVM/SVM.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0651dc674036b1ce"}},{"code_sha256_prefix":"6f81146fd4122ca8","entry":"get_model","repo":"UrosOgrizovic/SimpleGoogleQuickdraw","repo_kind":"listed","path":"models/transfer_learning/transfer_learning.py","file_url":"https://github.com/UrosOgrizovic/SimpleGoogleQuickdraw/blob/HEAD/models/transfer_learning/transfer_learning.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6f81146fd4122ca8"}},{"code_sha256_prefix":"a771fe43d3f305e2","entry":"get_model","repo":"UrosOgrizovic/SimpleGoogleQuickdraw","repo_kind":"listed","path":"models/vanilla_cnn/vanilla_cnn.py","file_url":"https://github.com/UrosOgrizovic/SimpleGoogleQuickdraw/blob/HEAD/models/vanilla_cnn/vanilla_cnn.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a771fe43d3f305e2"}},{"code_sha256_prefix":"0ab4dee93051e84f","entry":"load_image","repo":"krasserm/wdsr","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/krasserm/wdsr/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0ab4dee93051e84f"}},{"code_sha256_prefix":"2d732893414e2966","entry":"load_images","repo":"UrosOgrizovic/SimpleGoogleQuickdraw","repo_kind":"listed","path":"image_operations.py","file_url":"https://github.com/UrosOgrizovic/SimpleGoogleQuickdraw/blob/HEAD/image_operations.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2d732893414e2966"}},{"code_sha256_prefix":"ffc86641164a4445","entry":"pad_image","repo":"UrosOgrizovic/SimpleGoogleQuickdraw","repo_kind":"listed","path":"image_operations.py","file_url":"https://github.com/UrosOgrizovic/SimpleGoogleQuickdraw/blob/HEAD/image_operations.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ffc86641164a4445"}},{"code_sha256_prefix":"40c578a7c8a918e6","entry":"random_crop","repo":"krasserm/wdsr","repo_kind":"listed","path":"data.py","file_url":"https://github.com/krasserm/wdsr/blob/HEAD/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"40c578a7c8a918e6"}},{"code_sha256_prefix":"256bed53fab6520f","entry":"random_flip","repo":"krasserm/wdsr","repo_kind":"listed","path":"data.py","file_url":"https://github.com/krasserm/wdsr/blob/HEAD/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"256bed53fab6520f"}},{"code_sha256_prefix":"4cb58354ae7610d2","entry":"random_rotate","repo":"krasserm/wdsr","repo_kind":"listed","path":"data.py","file_url":"https://github.com/krasserm/wdsr/blob/HEAD/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4cb58354ae7610d2"}},{"code_sha256_prefix":"71ba60506facd18f","entry":"res_block","repo":"krasserm/wdsr","repo_kind":"listed","path":"model/edsr.py","file_url":"https://github.com/krasserm/wdsr/blob/HEAD/model/edsr.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"71ba60506facd18f"}},{"code_sha256_prefix":"fcb21bb87592664b","entry":"resolve","repo":"krasserm/wdsr","repo_kind":"listed","path":"model/common.py","file_url":"https://github.com/krasserm/wdsr/blob/HEAD/model/common.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fcb21bb87592664b"}},{"code_sha256_prefix":"ae235fb465d0b309","entry":"resolve_single","repo":"krasserm/wdsr","repo_kind":"listed","path":"model/common.py","file_url":"https://github.com/krasserm/wdsr/blob/HEAD/model/common.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ae235fb465d0b309"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}