{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/hypernetworks","title":"HyperNetworks","arxiv_id":"1609.09106","date":"2016-09-27","proceeding":null,"authors":["David Ha","Andrew Dai","Quoc V. Le"],"abstract":"This work explores hypernetworks: an approach of using a one network, also\nknown as a hypernetwork, to generate the weights for another network.\nHypernetworks provide an abstraction that is similar to what is found in\nnature: the relationship between a genotype - the hypernetwork - and a\nphenotype - the main network. Though they are also reminiscent of HyperNEAT in\nevolution, our hypernetworks are trained end-to-end with backpropagation and\nthus are usually faster. The focus of this work is to make hypernetworks useful\nfor deep convolutional networks and long recurrent networks, where\nhypernetworks can be viewed as relaxed form of weight-sharing across layers.\nOur main result is that hypernetworks can generate non-shared weights for LSTM\nand achieve near state-of-the-art results on a variety of sequence modelling\ntasks including character-level language modelling, handwriting generation and\nneural machine translation, challenging the weight-sharing paradigm for\nrecurrent networks. Our results also show that hypernetworks applied to\nconvolutional networks still achieve respectable results for image recognition\ntasks compared to state-of-the-art baseline models while requiring fewer\nlearnable parameters.","url_abs":"http://arxiv.org/abs/1609.09106v4","url_pdf":"http://arxiv.org/pdf/1609.09106v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"hypernetworks","repo_url":"https://github.com/cellistigs/ensemble_attention","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"hypernetworks","repo_url":"https://github.com/chrhenning/hypnettorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"hypernetworks","repo_url":"https://github.com/g1910/HyperNetworks","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"GPL-3.0"}},{"paper_slug":"hypernetworks","repo_url":"https://github.com/gahaalt/continual-learning-overview","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"hypernetworks","repo_url":"https://github.com/gahaalt/continual-learning-with-hypernets","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"hypernetworks","repo_url":"https://github.com/gtegner/hyper-gan","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"hypernetworks","repo_url":"https://github.com/pennfranc/hypnettorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"hypernetworks","repo_url":"https://github.com/shyamsn97/hyper-nn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":null},{"paper_slug":"hypernetworks","repo_url":"https://github.com/tjuhaoxiaotian/pymarl3","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"hypernetworks","repo_url":"https://github.com/labmlai/annotated_deep_learning_paper_implementations","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"handwriting-generation","task_name":"Handwriting generation"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"translation","task_name":"Translation"}],"methods":[{"method_slug":"hypernetwork","method_name":"HyperNetwork"}],"datasets_introduced":[],"methods_introduced":[{"slug":"hypernetwork","name":"HyperNetwork","full_name":"HyperNetwork"}],"results":[{"leaderboard":"/sota/language-modelling-on-penn-treebank-character","task":"Language Modelling","dataset":"Penn Treebank (Character Level)","model":"2-layer Norm HyperLSTM","rank_in_archive_order":14,"of":20,"metrics":{"Bit per Character (BPC)":"1.219","Number of params":"14.4M"},"uses_additional_data":false},{"leaderboard":"/sota/language-modelling-on-enwiki8","task":"Language Modelling","dataset":"enwik8","model":"Hypernetworks","rank_in_archive_order":40,"of":42,"metrics":{"Bit per Character (BPC)":"1.34","Number of params":"27M"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1609.09106","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1609.09106"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tjuhaoxiaotian/pymarl3","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gahaalt/continual-learning-overview","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shyamsn97/hyper-nn","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/g1910/HyperNetworks","reach":{"status":"ok","spdx":"GPL-3.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gahaalt/continual-learning-with-hypernets","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/chrhenning/hypnettorch","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gtegner/hyper-gan","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/labmlai/annotated_deep_learning_paper_implementations","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cellistigs/ensemble_attention","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pennfranc/hypnettorch","reach":{"status":"unanswered"}}],"summary":{"ran_draft_wrong":2,"ran_honours":1,"unverified":1},"by_repo_kind":{"listed":{"samples":4,"ran":3,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"8143f1c0cbd2ab4e","entry":"create_param_tree","repo":"shyamsn97/hyper-nn","repo_kind":"listed","path":"hypernn/jax/hypernet.py","file_url":"https://github.com/shyamsn97/hyper-nn/blob/HEAD/hypernn/jax/hypernet.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8143f1c0cbd2ab4e"}},{"code_sha256_prefix":"410618de180f045a","entry":"get_entropy","repo":"gtegner/hyper-gan","repo_kind":"listed","path":"hypergan_utils.py","file_url":"https://github.com/gtegner/hyper-gan/blob/HEAD/hypergan_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"410618de180f045a"}},{"code_sha256_prefix":"03e2fdfe356c35f0","entry":"true_data","repo":"gtegner/hyper-gan","repo_kind":"listed","path":"hypergan_utils.py","file_url":"https://github.com/gtegner/hyper-gan/blob/HEAD/hypergan_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"03e2fdfe356c35f0"}},{"code_sha256_prefix":"15ad5ae30ec820ea","entry":"plot_classification","repo":"gtegner/hyper-gan","repo_kind":"listed","path":"hypergan_utils.py","file_url":"https://github.com/gtegner/hyper-gan/blob/HEAD/hypergan_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"15ad5ae30ec820ea"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}