{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unified-embedding-battle-tested-feature","title":"Unified Embedding: Battle-Tested Feature Representations for Web-Scale ML Systems","arxiv_id":"2305.12102","date":"2023-05-20","proceeding":"NeurIPS 2023 11","authors":["Benjamin Coleman","Wang-Cheng Kang","Matthew Fahrbach","Ruoxi Wang","Lichan Hong","Ed H. Chi","Derek Zhiyuan Cheng"],"abstract":"Learning high-quality feature embeddings efficiently and effectively is critical for the performance of web-scale machine learning systems. A typical model ingests hundreds of features with vocabularies on the order of millions to billions of tokens. The standard approach is to represent each feature value as a d-dimensional embedding, introducing hundreds of billions of parameters for extremely high-cardinality features. This bottleneck has led to substantial progress in alternative embedding algorithms. Many of these methods, however, make the assumption that each feature uses an independent embedding table. This work introduces a simple yet highly effective framework, Feature Multiplexing, where one single representation space is used across many different categorical features. Our theoretical and empirical analysis reveals that multiplexed embeddings can be decomposed into components from each constituent feature, allowing models to distinguish between features. We show that multiplexed representations lead to Pareto-optimal parameter-accuracy tradeoffs for three public benchmark datasets. Further, we propose a highly practical approach called Unified Embedding with three major benefits: simplified feature configuration, strong adaptation to dynamic data distributions, and compatibility with modern hardware. Unified embedding gives significant improvements in offline and online metrics compared to highly competitive baselines across five web-scale search, ads, and recommender systems, where it serves billions of users across the world in industry-leading products.","url_abs":"https://arxiv.org/abs/2305.12102v3","url_pdf":"https://arxiv.org/pdf/2305.12102v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unified-embedding-battle-tested-feature","repo_url":"https://github.com/ledmaster/unified-embeddings","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"unified-embedding-battle-tested-feature","repo_url":"https://github.com/MindSpore-scientific/code-11/tree/main/UE-Unified-Embedding-Battle-Tested-Feature","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"unified-embedding-battle-tested-feature","repo_url":"https://github.com/mindspore-ai/contrib/tree/master/intern/UE-Unified-Embedding-Battle-Tested-Feature","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"unified-embedding-battle-tested-feature","repo_url":"https://github.com/pwc-1/Paper-9/tree/main/4/UE-Unified-Embedding-Battle-Tested-Feature","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"unified-embedding-battle-tested-feature","repo_url":"https://github.com/pwc-1/Paper-9/tree/main/7/UE-Unified-Embedding-Battle-Tested-Feature","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"recommendation-systems","task_name":"Recommendation Systems"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.12102","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.12102"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pwc-1/Paper-9/tree/main/4/UE-Unified-Embedding-Battle-Tested-Feature","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MindSpore-scientific/code-11/tree/main/UE-Unified-Embedding-Battle-Tested-Feature","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ledmaster/unified-embeddings","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mindspore-ai/contrib/tree/master/intern/UE-Unified-Embedding-Battle-Tested-Feature","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pwc-1/Paper-9/tree/main/7/UE-Unified-Embedding-Battle-Tested-Feature","reach":null}],"summary":{"ran":1,"ran_violates":1},"by_repo_kind":{"listed":{"samples":2,"ran":2,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"18e8c98b76298d14","entry":"UnifiedEmbedding","repo":"ledmaster/unified-embeddings","repo_kind":"listed","path":"ue.py","file_url":"https://github.com/ledmaster/unified-embeddings/blob/HEAD/ue.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"18e8c98b76298d14"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"mindspore-ai/contrib","repo_kind":"listed","path":"application/Adaptive-Nesterov-Momentum-Algorithm/adan.py","file_url":"https://github.com/mindspore-ai/contrib/blob/HEAD/application/Adaptive-Nesterov-Momentum-Algorithm/adan.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}