{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2510-22880","title":"Learning Reconfigurable Representations for Multimodal Federated Learning with Missing Data","arxiv_id":"2510.22880","date":"2025-10-27","proceeding":"NeurIPS","authors":["Duong M. Nguyen","Trong Nghia Hoang","Thanh Trung Huynh","Quoc Viet Hung Nguyen","Phi Le Nguyen"],"abstract":"Multimodal federated learning in real-world settings often encounters incomplete and heterogeneous data across clients. This results in misaligned local feature representations that limit the effectiveness of model aggregation. Unlike prior work that assumes either differing modality sets without missing input features or a shared modality set with missing features across clients, we consider a more general and realistic setting where each client observes a different subset of modalities and might also have missing input features within each modality. To address the resulting misalignment in learned representations, we propose a new federated learning framework featuring locally adaptive representations based on learnable client-side embedding controls that encode each client's data-missing patterns. These embeddings serve as reconfiguration signals that align the globally aggregated representation with each client's local context, enabling more effective use of shared information. Furthermore, the embedding controls can be algorithmically aggregated across clients with similar data-missing patterns to enhance the robustness of reconfiguration signals in adapting the global representation. Empirical results on multiple federated multimodal benchmarks with diverse data-missing patterns across clients demonstrate the efficacy of the proposed method, achieving up to 36.45\\% performance improvement under severe data incompleteness. The method is also supported by a theoretical analysis with an explicit performance bound that matches our empirical observations. Our source codes are provided at https://github.com/nmduonggg/PEPSY","url_abs":"https://arxiv.org/abs/2510.22880","url_pdf":"https://arxiv.org/pdf/2510.22880","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2510.22880","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2510.22880"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/nmduonggg/PEPSY","reach":null}],"summary":{"ran":11,"ran_draft_wrong":2,"unverified":15},"by_repo_kind":{"found_in_text":{"samples":28,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":28,"samples":[{"code_sha256_prefix":"393e5cd8f3952b7e","entry":"AggregationRebuild","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"393e5cd8f3952b7e"}},{"code_sha256_prefix":"ede801a4db754fbb","entry":"ContrastiveWeight","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ede801a4db754fbb"}},{"code_sha256_prefix":"e71d71067467b413","entry":"Inception1DBase","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e71d71067467b413"}},{"code_sha256_prefix":"55088101288bda31","entry":"_model_norm","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"55088101288bda31"}},{"code_sha256_prefix":"5e13db899f5d6014","entry":"_model_scale","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5e13db899f5d6014"}},{"code_sha256_prefix":"1536b4bb6ef52947","entry":"_model_sub","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1536b4bb6ef52947"}},{"code_sha256_prefix":"7e3423022c41a0cf","entry":"_modeldict_add","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7e3423022c41a0cf"}},{"code_sha256_prefix":"64c1a667680e0107","entry":"_modeldict_cossim","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"64c1a667680e0107"}},{"code_sha256_prefix":"392411e81ac8907b","entry":"_modeldict_dot","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"392411e81ac8907b"}},{"code_sha256_prefix":"c3ed27adaa2b4eda","entry":"_modeldict_norm","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c3ed27adaa2b4eda"}},{"code_sha256_prefix":"507bb3873b079432","entry":"_modeldict_scale","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"507bb3873b079432"}},{"code_sha256_prefix":"d04ee32603df82d1","entry":"_modeldict_sub","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d04ee32603df82d1"}},{"code_sha256_prefix":"755aa756a3b59a66","entry":"get_module_from_model","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"755aa756a3b59a66"}},{"code_sha256_prefix":"c9abe6c6f31a0292","entry":"Classifier","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c9abe6c6f31a0292"}},{"code_sha256_prefix":"86fa8e05c8d901b9","entry":"FLinear","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"86fa8e05c8d901b9"}},{"code_sha256_prefix":"6b254025739f9a8b","entry":"FModule","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6b254025739f9a8b"}},{"code_sha256_prefix":"ad2c6ad206c68936","entry":"GlobalFuser","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ad2c6ad206c68936"}},{"code_sha256_prefix":"6b6ddbf143579632","entry":"InceptionBlock1D","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6b6ddbf143579632"}},{"code_sha256_prefix":"3312f7e4eef62211","entry":"ModalityEncoder","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3312f7e4eef62211"}},{"code_sha256_prefix":"0571629de00b209f","entry":"Model","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0571629de00b209f"}},{"code_sha256_prefix":"bbebd4f48b55a6ae","entry":"Pool","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bbebd4f48b55a6ae"}},{"code_sha256_prefix":"9959df95b38dce30","entry":"Shortcut1D","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9959df95b38dce30"}},{"code_sha256_prefix":"d6f9ab66ed1431d8","entry":"SimilarityProjector","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d6f9ab66ed1431d8"}},{"code_sha256_prefix":"4192e11d469e62df","entry":"SpecificEmbedding","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4192e11d469e62df"}},{"code_sha256_prefix":"ba5fc6e54501a867","entry":"_model_add","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ba5fc6e54501a867"}},{"code_sha256_prefix":"be6a4d28fd1f95af","entry":"_model_cossim","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"be6a4d28fd1f95af"}},{"code_sha256_prefix":"67ec258ed0b6c975","entry":"_model_dot","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"67ec258ed0b6c975"}},{"code_sha256_prefix":"af7fba9f010baea5","entry":"_modeldict_cp","repo":"nmduonggg/PEPSY","repo_kind":"found_in_text","path":"src/benchmark/edf_classification_lm/model/pepsy.py","file_url":"https://github.com/nmduonggg/PEPSY/blob/HEAD/src/benchmark/edf_classification_lm/model/pepsy.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"af7fba9f010baea5"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_api"},"syntology_extracted_results":null}