{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/fusemoe-mixture-of-experts-transformers-for","title":"FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion","arxiv_id":"2402.03226","date":"2024-02-05","proceeding":null,"authors":["Xing Han","Huy Nguyen","Carl Harris","Nhat Ho","Suchi Saria"],"abstract":"As machine learning models in critical fields increasingly grapple with multimodal data, they face the dual challenges of handling a wide array of modalities, often incomplete due to missing elements, and the temporal irregularity and sparsity of collected samples. Successfully leveraging this complex data, while overcoming the scarcity of high-quality training samples, is key to improving these models' predictive performance. We introduce ``FuseMoE'', a mixture-of-experts framework incorporated with an innovative gating function. Designed to integrate a diverse number of modalities, FuseMoE is effective in managing scenarios with missing modalities and irregularly sampled data trajectories. Theoretically, our unique gating function contributes to enhanced convergence rates, leading to better performance in multiple downstream tasks. The practical utility of FuseMoE in the real world is validated by a diverse set of challenging prediction tasks.","url_abs":"https://arxiv.org/abs/2402.03226v3","url_pdf":"https://arxiv.org/pdf/2402.03226v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"missing-elements","task_name":"Missing Elements"},{"task_slug":"mixture-of-experts","task_name":"Mixture-of-Experts"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.03226","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.03226"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/aaronhan223/FuseMoE","reach":null}],"summary":{"ran":2,"ran_fixture":1,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":4,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":4,"samples":[{"code_sha256_prefix":"e59025cfd70fb4ec","entry":"Experts","repo":"aaronhan223/FuseMoE","repo_kind":"found_in_text","path":"src/core/hierarchical_moe.py","file_url":"https://github.com/aaronhan223/FuseMoE/blob/HEAD/src/core/hierarchical_moe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e59025cfd70fb4ec"}},{"code_sha256_prefix":"3263d32da10f25d5","entry":"Top2Gating","repo":"aaronhan223/FuseMoE","repo_kind":"found_in_text","path":"src/core/hierarchical_moe.py","file_url":"https://github.com/aaronhan223/FuseMoE/blob/HEAD/src/core/hierarchical_moe.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3263d32da10f25d5"}},{"code_sha256_prefix":"3f6b8710f20ce1cf","entry":"cumsum_exclusive","repo":"aaronhan223/FuseMoE","repo_kind":"found_in_text","path":"src/core/hierarchical_moe.py","file_url":"https://github.com/aaronhan223/FuseMoE/blob/HEAD/src/core/hierarchical_moe.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3f6b8710f20ce1cf"}},{"code_sha256_prefix":"0831d67669bf38e7","entry":"HierarchicalMoE","repo":"aaronhan223/FuseMoE","repo_kind":"found_in_text","path":"src/core/hierarchical_moe.py","file_url":"https://github.com/aaronhan223/FuseMoE/blob/HEAD/src/core/hierarchical_moe.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0831d67669bf38e7"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}