{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/disparse-disentangled-sparsification-for-1","title":"DiSparse: Disentangled Sparsification for Multitask Model Compression","arxiv_id":"2206.04662","date":"2022-06-09","proceeding":"CVPR 2022 1","authors":["Xinglong Sun","Ali Hassani","Zhangyang Wang","Gao Huang","Humphrey Shi"],"abstract":"Despite the popularity of Model Compression and Multitask Learning, how to effectively compress a multitask model has been less thoroughly analyzed due to the challenging entanglement of tasks in the parameter space. In this paper, we propose DiSparse, a simple, effective, and first-of-its-kind multitask pruning and sparse training scheme. We consider each task independently by disentangling the importance measurement and take the unanimous decisions among all tasks when performing parameter pruning and selection. Our experimental results demonstrate superior performance on various configurations and settings compared to popular sparse training and pruning methods. Besides the effectiveness in compression, DiSparse also provides a powerful tool to the multitask learning community. Surprisingly, we even observed better performance than some dedicated multitask learning methods in several cases despite the high model sparsity enforced by DiSparse. We analyzed the pruning masks generated with DiSparse and observed strikingly similar sparse network architecture identified by each task even before the training starts. We also observe the existence of a \"watershed\" layer where the task relatedness sharply drops, implying no benefits in continued parameters sharing. Our code and models will be available at: https://github.com/SHI-Labs/DiSparse-Multitask-Model-Compression.","url_abs":"https://arxiv.org/abs/2206.04662v1","url_pdf":"https://arxiv.org/pdf/2206.04662v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"disparse-disentangled-sparsification-for-1","repo_url":"https://github.com/shi-labs/disparse-multitask-model-compression","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"model-compression","task_name":"Model Compression"},{"task_slug":"model","task_name":"model"}],"methods":[{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2206.04662","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.04662"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/shi-labs/disparse-multitask-model-compression","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/SHI-Labs/DiSparse-Multitask-Model-Compression","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":2,"unverified":4},"by_repo_kind":{"official":{"samples":6,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"85b11b5ac1824b27","entry":"get_pruned_init","repo":"SHI-Labs/DiSparse-Multitask-Model-Compression","repo_kind":"official","path":"dynamic_prune.py","file_url":"https://github.com/SHI-Labs/DiSparse-Multitask-Model-Compression/blob/HEAD/dynamic_prune.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"85b11b5ac1824b27"}},{"code_sha256_prefix":"698e8939c8e2bde6","entry":"get_sparsity_dict","repo":"SHI-Labs/DiSparse-Multitask-Model-Compression","repo_kind":"official","path":"dynamic_prune.py","file_url":"https://github.com/SHI-Labs/DiSparse-Multitask-Model-Compression/blob/HEAD/dynamic_prune.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"698e8939c8e2bde6"}},{"code_sha256_prefix":"d753c50cd4fedd5b","entry":"deepcopy_pruned_net","repo":"SHI-Labs/DiSparse-Multitask-Model-Compression","repo_kind":"official","path":"dynamic_prune.py","file_url":"https://github.com/SHI-Labs/DiSparse-Multitask-Model-Compression/blob/HEAD/dynamic_prune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d753c50cd4fedd5b"}},{"code_sha256_prefix":"a1947097cace01b4","entry":"deepcopy_pruned_net_v2","repo":"SHI-Labs/DiSparse-Multitask-Model-Compression","repo_kind":"official","path":"dynamic_prune.py","file_url":"https://github.com/SHI-Labs/DiSparse-Multitask-Model-Compression/blob/HEAD/dynamic_prune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a1947097cace01b4"}},{"code_sha256_prefix":"0b70bdfc84ab571f","entry":"dynamic_disparse","repo":"SHI-Labs/DiSparse-Multitask-Model-Compression","repo_kind":"official","path":"dynamic_prune.py","file_url":"https://github.com/SHI-Labs/DiSparse-Multitask-Model-Compression/blob/HEAD/dynamic_prune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0b70bdfc84ab571f"}},{"code_sha256_prefix":"c9c7d2f187b2b92e","entry":"pseudo_forward","repo":"SHI-Labs/DiSparse-Multitask-Model-Compression","repo_kind":"official","path":"dynamic_prune.py","file_url":"https://github.com/SHI-Labs/DiSparse-Multitask-Model-Compression/blob/HEAD/dynamic_prune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c9c7d2f187b2b92e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}