{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/hllm-enhancing-sequential-recommendations-via","title":"HLLM: Enhancing Sequential Recommendations via Hierarchical Large Language Models for Item and User Modeling","arxiv_id":"2409.12740","date":"2024-09-19","proceeding":null,"authors":["Junyi Chen","Lu Chi","Bingyue Peng","Zehuan Yuan"],"abstract":"Large Language Models (LLMs) have achieved remarkable success in various fields, prompting several studies to explore their potential in recommendation systems. However, these attempts have so far resulted in only modest improvements over traditional recommendation models. Moreover, three critical questions remain under-explored: firstly, the real value of LLMs' pre-trained weights, often considered to encapsulate world knowledge; secondly, the necessity of fine-tuning for recommendation tasks; lastly, whether LLMs can exhibit the same scalability benefits in recommendation systems as they do in other domains. In this paper, we propose a novel Hierarchical Large Language Model (HLLM) architecture designed to enhance sequential recommendation systems. Our approach employs a two-tier model: the first Item LLM extracts rich content features from the detailed text description of the item, while the second User LLM utilizes these features to predict users' future interests based on their interaction history. Extensive experiments demonstrate that our method effectively leverages the pre-trained capabilities of open-source LLMs, and further fine-tuning leads to significant performance boosts. Additionally, HLLM achieves excellent scalability, with the largest configuration utilizing 7B parameters for both item feature extraction and user interest modeling. Moreover, HLLM offers excellent training and serving efficiency, making it practical in real-world applications. Evaluations on two large-scale datasets, PixelRec and Amazon Reviews, show that HLLM achieves state-of-the-art results, outperforming traditional ID-based models by a wide margin. In online A/B testing, HLLM showcases notable gains, validating its practical impact in real-world recommendation scenarios. Codes are available at https://github.com/bytedance/HLLM.","url_abs":"https://arxiv.org/abs/2409.12740v1","url_pdf":"https://arxiv.org/pdf/2409.12740v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"hllm-enhancing-sequential-recommendations-via","repo_url":"https://github.com/bytedance/hllm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"large-language-model","task_name":"Large Language Model"},{"task_slug":"recommendation-systems","task_name":"Recommendation Systems"},{"task_slug":"sequential-recommendation","task_name":"Sequential Recommendation"},{"task_slug":"world-knowledge","task_name":"World Knowledge"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2409.12740","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2409.12740"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bytedance/hllm","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":5,"unverified":2},"by_repo_kind":{"official":{"samples":7,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8c27099f9635928e","entry":"construct_rewrite_prompt_withuser_opensource","repo":"bytedance/hllm","repo_kind":"official","path":"code/HLLM_Creator_data_scripts/gpt_rewrite_user_open.py","file_url":"https://github.com/bytedance/hllm/blob/HEAD/code/HLLM_Creator_data_scripts/gpt_rewrite_user_open.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8c27099f9635928e"}},{"code_sha256_prefix":"9330158c89f4cd1f","entry":"extract_json_from_markdown","repo":"bytedance/hllm","repo_kind":"official","path":"code/HLLM_Creator_data_scripts/gpt_rewrite_user_open.py","file_url":"https://github.com/bytedance/hllm/blob/HEAD/code/HLLM_Creator_data_scripts/gpt_rewrite_user_open.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9330158c89f4cd1f"}},{"code_sha256_prefix":"261d4b69d11323bc","entry":"forward","repo":"bytedance/hllm","repo_kind":"official","path":"code/HLLM_Creator_eval_scripts/generate_useremb.py","file_url":"https://github.com/bytedance/hllm/blob/HEAD/code/HLLM_Creator_eval_scripts/generate_useremb.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"261d4b69d11323bc"}},{"code_sha256_prefix":"74bf1290122b46b9","entry":"gen_prompt2","repo":"bytedance/hllm","repo_kind":"official","path":"code/HLLM_Creator_data_scripts/gen_user_open.py","file_url":"https://github.com/bytedance/hllm/blob/HEAD/code/HLLM_Creator_data_scripts/gen_user_open.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"74bf1290122b46b9"}},{"code_sha256_prefix":"2003997271de811b","entry":"user_prompt_opensource","repo":"bytedance/hllm","repo_kind":"official","path":"code/HLLM_Creator_data_scripts/gpt_userinfo_open.py","file_url":"https://github.com/bytedance/hllm/blob/HEAD/code/HLLM_Creator_data_scripts/gpt_userinfo_open.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2003997271de811b"}},{"code_sha256_prefix":"bab644af7cd1ef26","entry":"forward","repo":"bytedance/hllm","repo_kind":"official","path":"code/HLLM_Creator_eval_scripts/generate_usertitle.py","file_url":"https://github.com/bytedance/hllm/blob/HEAD/code/HLLM_Creator_eval_scripts/generate_usertitle.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bab644af7cd1ef26"}},{"code_sha256_prefix":"a7f4c057cdcda01d","entry":"gen_prompt","repo":"bytedance/hllm","repo_kind":"official","path":"code/HLLM_Creator_eval_scripts/gpt_eval_open.py","file_url":"https://github.com/bytedance/hllm/blob/HEAD/code/HLLM_Creator_eval_scripts/gpt_eval_open.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a7f4c057cdcda01d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}