{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-brief-review-of-hypernetworks-in-deep","title":"A Brief Review of Hypernetworks in Deep Learning","arxiv_id":"2306.06955","date":"2023-06-12","proceeding":null,"authors":["Vinod Kumar Chauhan","Jiandong Zhou","Ping Lu","Soheila Molaei","David A. Clifton"],"abstract":"Hypernetworks, or hypernets for short, are neural networks that generate weights for another neural network, known as the target network. They have emerged as a powerful deep learning technique that allows for greater flexibility, adaptability, dynamism, faster training, information sharing, and model compression. Hypernets have shown promising results in a variety of deep learning problems, including continual learning, causal inference, transfer learning, weight pruning, uncertainty quantification, zero-shot learning, natural language processing, and reinforcement learning. Despite their success across different problem settings, there is currently no comprehensive review available to inform researchers about the latest developments and to assist in utilizing hypernets. To fill this gap, we review the progress in hypernets. We present an illustrative example of training deep neural networks using hypernets and propose categorizing hypernets based on five design criteria: inputs, outputs, variability of inputs and outputs, and the architecture of hypernets. We also review applications of hypernets across different deep learning problem settings, followed by a discussion of general scenarios where hypernets can be effectively employed. Finally, we discuss the challenges and future directions that remain underexplored in the field of hypernets. We believe that hypernetworks have the potential to revolutionize the field of deep learning. They offer a new way to design and train neural networks, and they have the potential to improve the performance of deep learning models on a variety of tasks. Through this review, we aim to inspire further advancements in deep learning through hypernetworks.","url_abs":"https://arxiv.org/abs/2306.06955v3","url_pdf":"https://arxiv.org/pdf/2306.06955v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-brief-review-of-hypernetworks-in-deep","repo_url":"https://github.com/jmdvinodjmd/HyperITE","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"causal-inference","task_name":"Causal Inference"},{"task_slug":"continual-learning","task_name":"Continual Learning"},{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"model-compression","task_name":"Model Compression"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"},{"task_slug":"uncertainty-quantification","task_name":"Uncertainty Quantification"},{"task_slug":"zero-shot-learning","task_name":"Zero-Shot Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2306.06955","atlas_url":"https://app.syntology.ai/?focus=2306.06955","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.06955"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jmdvinodjmd/HyperITE","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":12},"by_repo_kind":{"official":{"samples":12,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"43fc2f08683fd9a1","entry":"MLPFunctional","repo":"jmdvinodjmd/HyperITE","repo_kind":"official","path":"src/models/hn_utils.py","file_url":"https://github.com/jmdvinodjmd/HyperITE/blob/HEAD/src/models/hn_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"43fc2f08683fd9a1"}},{"code_sha256_prefix":"8dd582f3eaadaa64","entry":"calc_layer_param_max","repo":"jmdvinodjmd/HyperITE","repo_kind":"official","path":"src/models/hypernetworks.py","file_url":"https://github.com/jmdvinodjmd/HyperITE/blob/HEAD/src/models/hypernetworks.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8dd582f3eaadaa64"}},{"code_sha256_prefix":"b96b4b3c56d7f938","entry":"calc_metrics","repo":"jmdvinodjmd/HyperITE","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/jmdvinodjmd/HyperITE/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b96b4b3c56d7f938"}},{"code_sha256_prefix":"db491fe4e6a79756","entry":"calc_params","repo":"jmdvinodjmd/HyperITE","repo_kind":"official","path":"src/models/hypernetworks.py","file_url":"https://github.com/jmdvinodjmd/HyperITE/blob/HEAD/src/models/hypernetworks.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"db491fe4e6a79756"}},{"code_sha256_prefix":"1a4cab9a648afbdf","entry":"create_embeddings","repo":"jmdvinodjmd/HyperITE","repo_kind":"official","path":"src/models/hypernetworks.py","file_url":"https://github.com/jmdvinodjmd/HyperITE/blob/HEAD/src/models/hypernetworks.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1a4cab9a648afbdf"}},{"code_sha256_prefix":"e1d1253359cda097","entry":"get_dataset","repo":"jmdvinodjmd/HyperITE","repo_kind":"official","path":"src/load_model_data.py","file_url":"https://github.com/jmdvinodjmd/HyperITE/blob/HEAD/src/load_model_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e1d1253359cda097"}},{"code_sha256_prefix":"c8f7fcc00a2439dc","entry":"get_logger","repo":"jmdvinodjmd/HyperITE","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/jmdvinodjmd/HyperITE/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c8f7fcc00a2439dc"}},{"code_sha256_prefix":"4f0864055cb19cfa","entry":"get_loss","repo":"jmdvinodjmd/HyperITE","repo_kind":"official","path":"src/training_loop.py","file_url":"https://github.com/jmdvinodjmd/HyperITE/blob/HEAD/src/training_loop.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4f0864055cb19cfa"}},{"code_sha256_prefix":"2d6adf360f806515","entry":"get_meta_model","repo":"jmdvinodjmd/HyperITE","repo_kind":"official","path":"src/load_model_data.py","file_url":"https://github.com/jmdvinodjmd/HyperITE/blob/HEAD/src/load_model_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2d6adf360f806515"}},{"code_sha256_prefix":"75fcd30fa3971e9b","entry":"shapeWeights","repo":"jmdvinodjmd/HyperITE","repo_kind":"official","path":"src/models/hn_utils.py","file_url":"https://github.com/jmdvinodjmd/HyperITE/blob/HEAD/src/models/hn_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"75fcd30fa3971e9b"}},{"code_sha256_prefix":"f478ca191bec1a45","entry":"test","repo":"jmdvinodjmd/HyperITE","repo_kind":"official","path":"src/training_loop.py","file_url":"https://github.com/jmdvinodjmd/HyperITE/blob/HEAD/src/training_loop.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f478ca191bec1a45"}},{"code_sha256_prefix":"39cfd19e3f57f2b0","entry":"weighted_mse_loss","repo":"jmdvinodjmd/HyperITE","repo_kind":"official","path":"src/utils.py","file_url":"https://github.com/jmdvinodjmd/HyperITE/blob/HEAD/src/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"39cfd19e3f57f2b0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}