{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dynamic-slimmable-network","title":"Dynamic Slimmable Network","arxiv_id":"2103.13258","date":"2021-03-24","proceeding":"CVPR 2021 1","authors":["Changlin Li","Guangrun Wang","Bing Wang","Xiaodan Liang","Zhihui Li","Xiaojun Chang"],"abstract":"Current dynamic networks and dynamic pruning methods have shown their promising capability in reducing theoretical computation complexity. However, dynamic sparse patterns on convolutional filters fail to achieve actual acceleration in real-world implementation, due to the extra burden of indexing, weight-copying, or zero-masking. Here, we explore a dynamic network slimming regime, named Dynamic Slimmable Network (DS-Net), which aims to achieve good hardware-efficiency via dynamically adjusting filter numbers of networks at test time with respect to different inputs, while keeping filters stored statically and contiguously in hardware to prevent the extra burden. Our DS-Net is empowered with the ability of dynamic inference by the proposed double-headed dynamic gate that comprises an attention head and a slimming head to predictively adjust network width with negligible extra computation cost. To ensure generality of each candidate architecture and the fairness of gate, we propose a disentangled two-stage training scheme inspired by one-shot NAS. In the first stage, a novel training technique for weight-sharing networks named In-place Ensemble Bootstrapping is proposed to improve the supernet training efficacy. In the second stage, Sandwich Gate Sparsification is proposed to assist the gate training by identifying easy and hard samples in an online way. Extensive experiments demonstrate our DS-Net consistently outperforms its static counterparts as well as state-of-the-art static and dynamic model compression methods by a large margin (up to 5.9%). Typically, DS-Net achieves 2-4x computation reduction and 1.62x real-world acceleration over ResNet-50 and MobileNet with minimal accuracy drops on ImageNet. Code release: https://github.com/changlin31/DS-Net .","url_abs":"https://arxiv.org/abs/2103.13258v1","url_pdf":"https://arxiv.org/pdf/2103.13258v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dynamic-slimmable-network","repo_url":"https://github.com/changlin31/DS-Net","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"fairness","task_name":"Fairness"},{"task_slug":"model-compression","task_name":"Model Compression"}],"methods":[{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2103.13258","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2103.13258"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/changlin31/DS-Net","reach":null}],"summary":{"ran":5,"ran_fixture":1,"ran_draft_wrong":1,"unverified":10},"by_repo_kind":{"official":{"samples":17,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":17,"samples":[{"code_sha256_prefix":"5bd24c876f2a520e","entry":"DSAdaptiveAvgPool2d","repo":"changlin31/DS-Net","repo_kind":"official","path":"dyn_slim/models/dyn_slim_net.py","file_url":"https://github.com/changlin31/DS-Net/blob/HEAD/dyn_slim/models/dyn_slim_net.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5bd24c876f2a520e"}},{"code_sha256_prefix":"6741cb2067254f42","entry":"DSAvgPool2d","repo":"changlin31/DS-Net","repo_kind":"official","path":"dyn_slim/models/dyn_slim_net.py","file_url":"https://github.com/changlin31/DS-Net/blob/HEAD/dyn_slim/models/dyn_slim_net.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6741cb2067254f42"}},{"code_sha256_prefix":"f87f13d283307a0c","entry":"DSConv2d","repo":"changlin31/DS-Net","repo_kind":"official","path":"dyn_slim/models/dyn_slim_net.py","file_url":"https://github.com/changlin31/DS-Net/blob/HEAD/dyn_slim/models/dyn_slim_net.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f87f13d283307a0c"}},{"code_sha256_prefix":"62349f38e2e10903","entry":"DSpwConv2d","repo":"changlin31/DS-Net","repo_kind":"official","path":"dyn_slim/models/dyn_slim_net.py","file_url":"https://github.com/changlin31/DS-Net/blob/HEAD/dyn_slim/models/dyn_slim_net.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"62349f38e2e10903"}},{"code_sha256_prefix":"a86440e38234fea6","entry":"drop_path","repo":"changlin31/DS-Net","repo_kind":"official","path":"dyn_slim/models/dyn_slim_net.py","file_url":"https://github.com/changlin31/DS-Net/blob/HEAD/dyn_slim/models/dyn_slim_net.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a86440e38234fea6"}},{"code_sha256_prefix":"a385ad93fbda3711","entry":"gumbel_softmax","repo":"changlin31/DS-Net","repo_kind":"official","path":"dyn_slim/models/dyn_slim_net.py","file_url":"https://github.com/changlin31/DS-Net/blob/HEAD/dyn_slim/models/dyn_slim_net.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a385ad93fbda3711"}},{"code_sha256_prefix":"5b2c72c30130d62a","entry":"make_divisible","repo":"changlin31/DS-Net","repo_kind":"official","path":"dyn_slim/models/dyn_slim_net.py","file_url":"https://github.com/changlin31/DS-Net/blob/HEAD/dyn_slim/models/dyn_slim_net.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5b2c72c30130d62a"}},{"code_sha256_prefix":"cb0d38743570df22","entry":"DSBatchNorm2d","repo":"changlin31/DS-Net","repo_kind":"official","path":"dyn_slim/models/dyn_slim_net.py","file_url":"https://github.com/changlin31/DS-Net/blob/HEAD/dyn_slim/models/dyn_slim_net.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cb0d38743570df22"}},{"code_sha256_prefix":"7b461affe61c6107","entry":"DSDepthwiseSeparable","repo":"changlin31/DS-Net","repo_kind":"official","path":"dyn_slim/models/dyn_slim_net.py","file_url":"https://github.com/changlin31/DS-Net/blob/HEAD/dyn_slim/models/dyn_slim_net.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7b461affe61c6107"}},{"code_sha256_prefix":"712da76f69049631","entry":"DSInvertedResidual","repo":"changlin31/DS-Net","repo_kind":"official","path":"dyn_slim/models/dyn_slim_net.py","file_url":"https://github.com/changlin31/DS-Net/blob/HEAD/dyn_slim/models/dyn_slim_net.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"712da76f69049631"}},{"code_sha256_prefix":"183fb0ba92c58751","entry":"DSLinear","repo":"changlin31/DS-Net","repo_kind":"official","path":"dyn_slim/models/dyn_slim_net.py","file_url":"https://github.com/changlin31/DS-Net/blob/HEAD/dyn_slim/models/dyn_slim_net.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"183fb0ba92c58751"}},{"code_sha256_prefix":"5e7e8474749b7a63","entry":"DSNet","repo":"changlin31/DS-Net","repo_kind":"official","path":"dyn_slim/models/dyn_slim_net.py","file_url":"https://github.com/changlin31/DS-Net/blob/HEAD/dyn_slim/models/dyn_slim_net.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5e7e8474749b7a63"}},{"code_sha256_prefix":"5d30f606578e8871","entry":"DSStage","repo":"changlin31/DS-Net","repo_kind":"official","path":"dyn_slim/models/dyn_slim_net.py","file_url":"https://github.com/changlin31/DS-Net/blob/HEAD/dyn_slim/models/dyn_slim_net.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5d30f606578e8871"}},{"code_sha256_prefix":"e6fc22bbfc4c8edf","entry":"DSdwConv2d","repo":"changlin31/DS-Net","repo_kind":"official","path":"dyn_slim/models/dyn_slim_net.py","file_url":"https://github.com/changlin31/DS-Net/blob/HEAD/dyn_slim/models/dyn_slim_net.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e6fc22bbfc4c8edf"}},{"code_sha256_prefix":"4907ed2eb06578b4","entry":"MultiHeadGate","repo":"changlin31/DS-Net","repo_kind":"official","path":"dyn_slim/models/dyn_slim_net.py","file_url":"https://github.com/changlin31/DS-Net/blob/HEAD/dyn_slim/models/dyn_slim_net.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4907ed2eb06578b4"}},{"code_sha256_prefix":"5b52cc50d4717222","entry":"apply_differentiable_gate_channel","repo":"changlin31/DS-Net","repo_kind":"official","path":"dyn_slim/models/dyn_slim_net.py","file_url":"https://github.com/changlin31/DS-Net/blob/HEAD/dyn_slim/models/dyn_slim_net.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5b52cc50d4717222"}},{"code_sha256_prefix":"87214df03908dc48","entry":"set_exist_attr","repo":"changlin31/DS-Net","repo_kind":"official","path":"dyn_slim/models/dyn_slim_net.py","file_url":"https://github.com/changlin31/DS-Net/blob/HEAD/dyn_slim/models/dyn_slim_net.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"87214df03908dc48"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}