{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/language-models-encode-collaborative-signals","title":"Language Representations Can be What Recommenders Need: Findings and Potentials","arxiv_id":"2407.05441","date":"2024-07-07","proceeding":null,"authors":["Leheng Sheng","An Zhang","Yi Zhang","Yuxin Chen","Xiang Wang","Tat-Seng Chua"],"abstract":"Recent studies empirically indicate that language models (LMs) encode rich world knowledge beyond mere semantics, attracting significant attention across various fields. However, in the recommendation domain, it remains uncertain whether LMs implicitly encode user preference information. Contrary to prevailing understanding that LMs and traditional recommenders learn two distinct representation spaces due to the huge gap in language and behavior modeling objectives, this work re-examines such understanding and explores extracting a recommendation space directly from the language representation space. Surprisingly, our findings demonstrate that item representations, when linearly mapped from advanced LM representations, yield superior recommendation performance. This outcome suggests the possible homomorphism between the advanced language representation space and an effective item representation space for recommendation, implying that collaborative signals may be implicitly encoded within LMs. Motivated by these findings, we explore the possibility of designing advanced collaborative filtering (CF) models purely based on language representations without ID-based embeddings. To be specific, we incorporate several crucial components to build a simple yet effective model, with item titles as the input. Empirical results show that such a simple model can outperform leading ID-based CF models, which sheds light on using language representations for better recommendation. Moreover, we systematically analyze this simple model and find several key features for using advanced language representations: a good initialization for item representations, zero-shot recommendation abilities, and being aware of user intention. Our findings highlight the connection between language modeling and behavior modeling, which can inspire both natural language processing and recommender system communities.","url_abs":"https://arxiv.org/abs/2407.05441v4","url_pdf":"https://arxiv.org/pdf/2407.05441v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"language-models-encode-collaborative-signals","repo_url":"https://github.com/lehengthu/alpharec","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"collaborative-filtering","task_name":"Collaborative Filtering"},{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"recommendation-systems","task_name":"Recommendation Systems"},{"task_slug":"world-knowledge","task_name":"World Knowledge"}],"methods":[{"method_slug":"aware","method_name":"AWARE"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.05441","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.05441"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lehengthu/alpharec","reach":{"status":"ok"}}],"summary":{"ran":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"029327c4f6fb55d0","entry":"helper_load","repo":"lehengthu/alpharec","repo_kind":"official","path":"models/General/base/abstract_data.py","file_url":"https://github.com/lehengthu/alpharec/blob/HEAD/models/General/base/abstract_data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"029327c4f6fb55d0"}},{"code_sha256_prefix":"f1fe817c737b6fed","entry":"helper_load_train","repo":"lehengthu/alpharec","repo_kind":"official","path":"models/General/base/abstract_data.py","file_url":"https://github.com/lehengthu/alpharec/blob/HEAD/models/General/base/abstract_data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f1fe817c737b6fed"}},{"code_sha256_prefix":"ea36ff95c1499fb1","entry":"naive_sparse2tensor","repo":"lehengthu/alpharec","repo_kind":"official","path":"models/General/MultVAE.py","file_url":"https://github.com/lehengthu/alpharec/blob/HEAD/models/General/MultVAE.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ea36ff95c1499fb1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}