{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/datamux-data-multiplexing-for-neural-networks","title":"DataMUX: Data Multiplexing for Neural Networks","arxiv_id":"2202.09318","date":"2022-02-18","proceeding":null,"authors":["Vishvak Murahari","Carlos E. Jimenez","Runzhe Yang","Karthik Narasimhan"],"abstract":"In this paper, we introduce data multiplexing (DataMUX), a technique that enables deep neural networks to process multiple inputs simultaneously using a single compact representation. DataMUX demonstrates that neural networks are capable of generating accurate predictions over mixtures of inputs, resulting in increased throughput with minimal extra memory requirements. Our approach uses two key components -- 1) a multiplexing layer that performs a fixed linear transformation to each input before combining them to create a mixed representation of the same size as a single input, which is then processed by the base network, and 2) a demultiplexing layer that converts the base network's output back into independent representations before producing predictions for each input. We show the viability of DataMUX for different architectures (Transformers, and to a lesser extent MLPs and CNNs) across six different tasks spanning sentence classification, named entity recognition and image classification. For instance, DataMUX for Transformers can multiplex up to $20$x/$40$x inputs, achieving $11$x/$18$x increase in throughput with minimal absolute performance drops of $<2\\%$ and $<4\\%$ respectively on MNLI, a natural language inference task. We also provide a theoretical construction for multiplexing in self-attention networks and analyze the effect of various design elements in DataMUX.","url_abs":"https://arxiv.org/abs/2202.09318v2","url_pdf":"https://arxiv.org/pdf/2202.09318v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"datamux-data-multiplexing-for-neural-networks","repo_url":"https://github.com/princeton-nlp/datamux","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"named-entity-recognition-1","task_name":"Named Entity Recognition"},{"task_slug":"named-entity-recognition-ner","task_name":"Named Entity Recognition (NER)"},{"task_slug":"natural-language-inference","task_name":"Natural Language Inference"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"sentence-classification","task_name":"Sentence Classification"},{"task_slug":"image-classification","task_name":"image-classification"},{"task_slug":"named-entity-recognition","task_name":"named-entity-recognition"}],"methods":[{"method_slug":"base","method_name":"BASE"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2202.09318","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2202.09318"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/princeton-nlp/datamux","reach":null}],"summary":{"ran":6,"ran_honours":2,"unverified":1},"by_repo_kind":{"official":{"samples":9,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"4573eece1b643ae9","entry":"RetrievalHeadIndexDemultiplexing","repo":"princeton-nlp/datamux","repo_kind":"official","path":"models/multiplexing.py","file_url":"https://github.com/princeton-nlp/datamux/blob/HEAD/models/multiplexing.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"4573eece1b643ae9"}},{"code_sha256_prefix":"1f9ba0adc7fd6702","entry":"RetrievalHeadMLPDemultiplexing","repo":"princeton-nlp/datamux","repo_kind":"official","path":"models/multiplexing.py","file_url":"https://github.com/princeton-nlp/datamux/blob/HEAD/models/multiplexing.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"1f9ba0adc7fd6702"}},{"code_sha256_prefix":"546ff05a58f372a4","entry":"RobertaIndexDemultiplexerSequenceClassification","repo":"princeton-nlp/datamux","repo_kind":"official","path":"models/multiplexing.py","file_url":"https://github.com/princeton-nlp/datamux/blob/HEAD/models/multiplexing.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"546ff05a58f372a4"}},{"code_sha256_prefix":"807966683fd8b592","entry":"RobertaMLPDemultiplexerSequenceClassification","repo":"princeton-nlp/datamux","repo_kind":"official","path":"models/multiplexing.py","file_url":"https://github.com/princeton-nlp/datamux/blob/HEAD/models/multiplexing.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"807966683fd8b592"}},{"code_sha256_prefix":"cfe0543d2ad4dc5f","entry":"RobertaMLPDemuxModule","repo":"princeton-nlp/datamux","repo_kind":"official","path":"models/multiplexing.py","file_url":"https://github.com/princeton-nlp/datamux/blob/HEAD/models/multiplexing.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"cfe0543d2ad4dc5f"}},{"code_sha256_prefix":"165182edd4643caa","entry":"SequenceClassifierOutputMuxed","repo":"princeton-nlp/datamux","repo_kind":"official","path":"models/multiplexing.py","file_url":"https://github.com/princeton-nlp/datamux/blob/HEAD/models/multiplexing.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"165182edd4643caa"}},{"code_sha256_prefix":"181f1d62d685220a","entry":"binary_encoding","repo":"princeton-nlp/datamux","repo_kind":"official","path":"models/multiplexing.py","file_url":"https://github.com/princeton-nlp/datamux/blob/HEAD/models/multiplexing.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"181f1d62d685220a"}},{"code_sha256_prefix":"8e9f043a6aacef9a","entry":"random_encoding","repo":"princeton-nlp/datamux","repo_kind":"official","path":"models/multiplexing.py","file_url":"https://github.com/princeton-nlp/datamux/blob/HEAD/models/multiplexing.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"8e9f043a6aacef9a"}},{"code_sha256_prefix":"8a784cafcc4e2bf4","entry":"RobertaSequenceClassificationMuxed","repo":"princeton-nlp/datamux","repo_kind":"official","path":"models/multiplexing.py","file_url":"https://github.com/princeton-nlp/datamux/blob/HEAD/models/multiplexing.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"8a784cafcc4e2bf4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}