{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/efficient-parametrization-of-multi-domain","title":"Efficient parametrization of multi-domain deep neural networks","arxiv_id":"1803.10082","date":"2018-03-27","proceeding":"CVPR 2018 6","authors":["Sylvestre-Alvise Rebuffi","Hakan Bilen","Andrea Vedaldi"],"abstract":"A practical limitation of deep neural networks is their high degree of\nspecialization to a single task and visual domain. Recently, inspired by the\nsuccesses of transfer learning, several authors have proposed to learn instead\nuniversal, fixed feature extractors that, used as the first stage of any deep\nnetwork, work well for several tasks and domains simultaneously. Nevertheless,\nsuch universal features are still somewhat inferior to specialized networks.\n  To overcome this limitation, in this paper we propose to consider instead\nuniversal parametric families of neural networks, which still contain\nspecialized problem-specific models, but differing only by a small number of\nparameters. We study different designs for such parametrizations, including\nseries and parallel residual adapters, joint adapter compression, and parameter\nallocations, and empirically identify the ones that yield the highest\ncompression. We show that, in order to maximize performance, it is necessary to\nadapt both shallow and deep layers of a deep network, but the required changes\nare very small. We also show that these universal parametrization are very\neffective for transfer learning, where they outperform traditional fine-tuning\ntechniques.","url_abs":"http://arxiv.org/abs/1803.10082v1","url_pdf":"http://arxiv.org/pdf/1803.10082v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"efficient-parametrization-of-multi-domain","repo_url":"https://github.com/SLrepo/residual_adapter","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"efficient-parametrization-of-multi-domain","repo_url":"https://github.com/lukashedegaard/structured-pruning-adapters","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"efficient-parametrization-of-multi-domain","repo_url":"https://github.com/srebuffi/residual_adapters","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"continual-learning","task_name":"Continual Learning"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/continual-learning-on-visual-domain-decathlon","task":"Continual Learning","dataset":"visual domain decathlon (10 tasks)","model":"Parallel Res. adapt.","rank_in_archive_order":3,"of":14,"metrics":{"Avg. Accuracy":"78.07","decathlon discipline (Score)":"3412"},"uses_additional_data":false},{"leaderboard":"/sota/continual-learning-on-visual-domain-decathlon","task":"Continual Learning","dataset":"visual domain decathlon (10 tasks)","model":"Series Res. adapt.","rank_in_archive_order":5,"of":14,"metrics":{"decathlon discipline (Score)":"3159"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1803.10082","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1803.10082"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/SLrepo/residual_adapter","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lukashedegaard/structured-pruning-adapters","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/srebuffi/residual_adapters","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":1,"ran":1,"unverified":1},"by_repo_kind":{"listed":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"583f9780bdd00a45","entry":"conv3x3","repo":"srebuffi/residual_adapters","repo_kind":"listed","path":"models.py","file_url":"https://github.com/srebuffi/residual_adapters/blob/HEAD/models.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"583f9780bdd00a45"}},{"code_sha256_prefix":"534e80dab769ec2f","entry":"pil_loader","repo":"srebuffi/residual_adapters","repo_kind":"listed","path":"imdbfolder_coco.py","file_url":"https://github.com/srebuffi/residual_adapters/blob/HEAD/imdbfolder_coco.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"534e80dab769ec2f"}},{"code_sha256_prefix":"143aa84e26aba5c8","entry":"conv1x1_fonc","repo":"srebuffi/residual_adapters","repo_kind":"listed","path":"models.py","file_url":"https://github.com/srebuffi/residual_adapters/blob/HEAD/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"143aa84e26aba5c8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}