{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/decomposing-the-neurons-activation-sparsity","title":"Decomposing the Neurons: Activation Sparsity via Mixture of Experts for Continual Test Time Adaptation","arxiv_id":"2405.16486","date":"2024-05-26","proceeding":null,"authors":["Rongyu Zhang","Aosong Cheng","Yulin Luo","Gaole Dai","Huanrui Yang","Jiaming Liu","ran Xu","Li Du","Yuan Du","Yanbing Jiang","Shanghang Zhang"],"abstract":"Continual Test-Time Adaptation (CTTA), which aims to adapt the pre-trained model to ever-evolving target domains, emerges as an important task for vision models. As current vision models appear to be heavily biased towards texture, continuously adapting the model from one domain distribution to another can result in serious catastrophic forgetting. Drawing inspiration from the human visual system's adeptness at processing both shape and texture according to the famous Trichromatic Theory, we explore the integration of a Mixture-of-Activation-Sparsity-Experts (MoASE) as an adapter for the CTTA task. Given the distinct reaction of neurons with low/high activation to domain-specific/agnostic features, MoASE decomposes the neural activation into high-activation and low-activation components with a non-differentiable Spatial Differentiate Dropout (SDD). Based on the decomposition, we devise a multi-gate structure comprising a Domain-Aware Gate (DAG) that utilizes domain information to adaptive combine experts that process the post-SDD sparse activations of different strengths, and the Activation Sparsity Gate (ASG) that adaptively assigned feature selection threshold of the SDD for different experts for more precise feature decomposition. Finally, we introduce a Homeostatic-Proximal (HP) loss to bypass the error accumulation problem when continuously adapting the model. Extensive experiments on four prominent benchmarks substantiate that our methodology achieves state-of-the-art performance in both classification and segmentation CTTA tasks. Our code is now available at https://github.com/RoyZry98/MoASE-Pytorch.","url_abs":"https://arxiv.org/abs/2405.16486v1","url_pdf":"https://arxiv.org/pdf/2405.16486v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"decomposing-the-neurons-activation-sparsity","repo_url":"https://github.com/royzry98/moase-pytorch","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"mixture-of-experts","task_name":"Mixture-of-Experts"},{"task_slug":"test-time-adaptation","task_name":"Test-time Adaptation"},{"task_slug":"feature-selection","task_name":"feature selection"}],"methods":[{"method_slug":"adapter","method_name":"Adapter"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"feature-selection","method_name":"Feature Selection"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2405.16486","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.16486"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/royzry98/moase-pytorch","reach":{"status":"ok"}}],"summary":{"ran_violates":1,"ran":7,"ran_honours":1,"ran_draft_wrong":2,"unverified":4},"by_repo_kind":{"official":{"samples":15,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":15,"samples":[{"code_sha256_prefix":"b1f9cebf0d177140","entry":"boolean_string","repo":"royzry98/moase-pytorch","repo_kind":"official","path":"utils.py","file_url":"https://github.com/royzry98/moase-pytorch/blob/HEAD/utils.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b1f9cebf0d177140"}},{"code_sha256_prefix":"c5f915703f30ab89","entry":"collect_stats","repo":"royzry98/moase-pytorch","repo_kind":"official","path":"norm.py","file_url":"https://github.com/royzry98/moase-pytorch/blob/HEAD/norm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c5f915703f30ab89"}},{"code_sha256_prefix":"8f537ee8d4a9b397","entry":"configure_model","repo":"royzry98/moase-pytorch","repo_kind":"official","path":"norm.py","file_url":"https://github.com/royzry98/moase-pytorch/blob/HEAD/norm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8f537ee8d4a9b397"}},{"code_sha256_prefix":"18b41dffa7b33f34","entry":"convert_tensor_to_image","repo":"royzry98/moase-pytorch","repo_kind":"official","path":"utils.py","file_url":"https://github.com/royzry98/moase-pytorch/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"18b41dffa7b33f34"}},{"code_sha256_prefix":"4aedf5bc9100546f","entry":"make_custom_dataset","repo":"royzry98/moase-pytorch","repo_kind":"official","path":"robustbench/loaders.py","file_url":"https://github.com/royzry98/moase-pytorch/blob/HEAD/robustbench/loaders.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4aedf5bc9100546f"}},{"code_sha256_prefix":"660527f45da7af3b","entry":"param_augment","repo":"royzry98/moase-pytorch","repo_kind":"official","path":"cotta_vit_opd.py","file_url":"https://github.com/royzry98/moase-pytorch/blob/HEAD/cotta_vit_opd.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"660527f45da7af3b"}},{"code_sha256_prefix":"f321f54723433661","entry":"pil_loader","repo":"royzry98/moase-pytorch","repo_kind":"official","path":"robustbench/loaders.py","file_url":"https://github.com/royzry98/moase-pytorch/blob/HEAD/robustbench/loaders.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f321f54723433661"}},{"code_sha256_prefix":"02d40fa8eb9d95d7","entry":"pytorch_evaluate","repo":"royzry98/moase-pytorch","repo_kind":"official","path":"utils.py","file_url":"https://github.com/royzry98/moase-pytorch/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"02d40fa8eb9d95d7"}},{"code_sha256_prefix":"6bc4cd7ae8f4b04d","entry":"reverse_kl","repo":"royzry98/moase-pytorch","repo_kind":"official","path":"cotta_vit_opd.py","file_url":"https://github.com/royzry98/moase-pytorch/blob/HEAD/cotta_vit_opd.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6bc4cd7ae8f4b04d"}},{"code_sha256_prefix":"721e565ebd5fda4e","entry":"setup_source","repo":"royzry98/moase-pytorch","repo_kind":"official","path":"cifar100c_vit.py","file_url":"https://github.com/royzry98/moase-pytorch/blob/HEAD/cifar100c_vit.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"721e565ebd5fda4e"}},{"code_sha256_prefix":"a77d556c0aea19c1","entry":"update_ema_variables","repo":"royzry98/moase-pytorch","repo_kind":"official","path":"cotta_vit.py","file_url":"https://github.com/royzry98/moase-pytorch/blob/HEAD/cotta_vit.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a77d556c0aea19c1"}},{"code_sha256_prefix":"404fb2b2daa1ae78","entry":"accimage_loader","repo":"royzry98/moase-pytorch","repo_kind":"official","path":"robustbench/loaders.py","file_url":"https://github.com/royzry98/moase-pytorch/blob/HEAD/robustbench/loaders.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"404fb2b2daa1ae78"}},{"code_sha256_prefix":"2ef65c34f0fa8042","entry":"load_cifar10","repo":"royzry98/moase-pytorch","repo_kind":"official","path":"robustbench/data.py","file_url":"https://github.com/royzry98/moase-pytorch/blob/HEAD/robustbench/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2ef65c34f0fa8042"}},{"code_sha256_prefix":"52acd5b81cee97b1","entry":"load_cifar100","repo":"royzry98/moase-pytorch","repo_kind":"official","path":"robustbench/data.py","file_url":"https://github.com/royzry98/moase-pytorch/blob/HEAD/robustbench/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"52acd5b81cee97b1"}},{"code_sha256_prefix":"e589e47d748af885","entry":"softmax_entropy","repo":"royzry98/moase-pytorch","repo_kind":"official","path":"cotta_vit.py","file_url":"https://github.com/royzry98/moase-pytorch/blob/HEAD/cotta_vit.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e589e47d748af885"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}