{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/long-sequence-recommendation-models-need","title":"Long-Sequence Recommendation Models Need Decoupled Embeddings","arxiv_id":"2410.02604","date":"2024-10-03","proceeding":null,"authors":["Ningya Feng","Junwei Pan","Jialong Wu","Baixu Chen","Ximei Wang","Qian Li","Xian Hu","Jie Jiang","Mingsheng Long"],"abstract":"Lifelong user behavior sequences are crucial for capturing user interests and predicting user responses in modern recommendation systems. A two-stage paradigm is typically adopted to handle these long sequences: a subset of relevant behaviors is first searched from the original long sequences via an attention mechanism in the first stage and then aggregated with the target item to construct a discriminative representation for prediction in the second stage. In this work, we identify and characterize, for the first time, a neglected deficiency in existing long-sequence recommendation models: a single set of embeddings struggles with learning both attention and representation, leading to interference between these two processes. Initial attempts to address this issue with some common methods (e.g., linear projections -- a technique borrowed from language processing) proved ineffective, shedding light on the unique challenges of recommendation models. To overcome this, we propose the Decoupled Attention and Representation Embeddings (DARE) model, where two distinct embedding tables are initialized and learned separately to fully decouple attention and representation. Extensive experiments and analysis demonstrate that DARE provides more accurate searches of correlated behaviors and outperforms baselines with AUC gains up to 0.9% on public datasets and notable improvements on Tencent's advertising platform. Furthermore, decoupling embedding spaces allows us to reduce the attention embedding dimension and accelerate the search procedure by 50% without significant performance impact, enabling more efficient, high-performance online serving. Code in PyTorch for experiments, including model analysis, is available at https://github.com/thuml/DARE.","url_abs":"https://arxiv.org/abs/2410.02604v3","url_pdf":"https://arxiv.org/pdf/2410.02604v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"long-sequence-recommendation-models-need","repo_url":"https://github.com/thuml/dare","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"recommendation-systems","task_name":"Recommendation Systems"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"set","method_name":"SET"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.02604","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.02604"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/thuml/DARE","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/thuml/dare","reach":{"status":"ok"}}],"summary":{"ran":8,"ran_violates":1},"by_repo_kind":{"official":{"samples":9,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"5f13f35abd14dc82","entry":"AttentionModel","repo":"thuml/dare","repo_kind":"official","path":"model/base_model_pytorch.py","file_url":"https://github.com/thuml/dare/blob/HEAD/model/base_model_pytorch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5f13f35abd14dc82"}},{"code_sha256_prefix":"aa3f1f6def5c6e54","entry":"BaseModel","repo":"thuml/dare","repo_kind":"official","path":"model/base_model_pytorch.py","file_url":"https://github.com/thuml/dare/blob/HEAD/model/base_model_pytorch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"aa3f1f6def5c6e54"}},{"code_sha256_prefix":"b5459f1f6325b000","entry":"attention","repo":"thuml/DARE","repo_kind":"official","path":"model/attention_pytorch.py","file_url":"https://github.com/thuml/DARE/blob/HEAD/model/attention_pytorch.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b5459f1f6325b000"}},{"code_sha256_prefix":"7873f0e1fa137cda","entry":"calc_auc","repo":"thuml/DARE","repo_kind":"official","path":"utils.py","file_url":"https://github.com/thuml/DARE/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7873f0e1fa137cda"}},{"code_sha256_prefix":"f6dbcc7447051099","entry":"gauss_normalize","repo":"thuml/DARE","repo_kind":"official","path":"analysis/attention_accuracy_analysis/calc_learned.py","file_url":"https://github.com/thuml/DARE/blob/HEAD/analysis/attention_accuracy_analysis/calc_learned.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f6dbcc7447051099"}},{"code_sha256_prefix":"ef9170abf085609d","entry":"min_max_normalize","repo":"thuml/DARE","repo_kind":"official","path":"analysis/attention_accuracy_analysis/calc_learned.py","file_url":"https://github.com/thuml/DARE/blob/HEAD/analysis/attention_accuracy_analysis/calc_learned.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ef9170abf085609d"}},{"code_sha256_prefix":"86c7147edb17dd5d","entry":"normalization","repo":"thuml/dare","repo_kind":"official","path":"analysis/attention_accuracy_analysis/calc_gt.py","file_url":"https://github.com/thuml/dare/blob/HEAD/analysis/attention_accuracy_analysis/calc_gt.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"86c7147edb17dd5d"}},{"code_sha256_prefix":"9e51ba90ca7253fa","entry":"pad_and_truncate","repo":"thuml/DARE","repo_kind":"official","path":"preprocess/taobao_v4.py","file_url":"https://github.com/thuml/DARE/blob/HEAD/preprocess/taobao_v4.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9e51ba90ca7253fa"}},{"code_sha256_prefix":"a585b44308980803","entry":"softmax","repo":"thuml/DARE","repo_kind":"official","path":"analysis/attention_accuracy_analysis/calc_learned.py","file_url":"https://github.com/thuml/DARE/blob/HEAD/analysis/attention_accuracy_analysis/calc_learned.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a585b44308980803"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}