{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/recflow-an-industrial-full-flow","title":"RecFlow: An Industrial Full Flow Recommendation Dataset","arxiv_id":"2410.20868","date":"2024-10-28","proceeding":null,"authors":["Qi Liu","Kai Zheng","Rui Huang","Wuchao Li","Kuo Cai","Yuan Chai","Yanan Niu","Yiqun Hui","Bing Han","Na Mou","Hongning Wang","Wentian Bao","Yunen Yu","Guorui Zhou","Han Li","Yang song","Defu Lian","Kun Gai"],"abstract":"Industrial recommendation systems (RS) rely on the multi-stage pipeline to balance effectiveness and efficiency when delivering items from a vast corpus to users. Existing RS benchmark datasets primarily focus on the exposure space, where novel RS algorithms are trained and evaluated. However, when these algorithms transition to real world industrial RS, they face a critical challenge of handling unexposed items which are a significantly larger space than the exposed one. This discrepancy profoundly impacts their practical performance. Additionally, these algorithms often overlook the intricate interplay between multiple RS stages, resulting in suboptimal overall system performance. To address this issue, we introduce RecFlow, an industrial full flow recommendation dataset designed to bridge the gap between offline RS benchmarks and the real online environment. Unlike existing datasets, RecFlow includes samples not only from the exposure space but also unexposed items filtered at each stage of the RS funnel. Our dataset comprises 38M interactions from 42K users across nearly 9M items with additional 1.9B stage samples collected from 9.3M online requests over 37 days and spanning 6 stages. Leveraging the RecFlow dataset, we conduct courageous exploration experiments, showcasing its potential in designing new algorithms to enhance effectiveness by incorporating stage-specific samples. Some of these algorithms have already been deployed online, consistently yielding significant gains. We propose RecFlow as the first comprehensive benchmark dataset for the RS community, supporting research on designing algorithms at any stage, study of selection bias, debiased algorithms, multi-stage consistency and optimality, multi-task recommendation, and user behavior modeling. The RecFlow dataset, along with the corresponding source code, is available at https://github.com/RecFlow-ICLR/RecFlow.","url_abs":"https://arxiv.org/abs/2410.20868v1","url_pdf":"https://arxiv.org/pdf/2410.20868v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"recflow-an-industrial-full-flow","repo_url":"https://github.com/recflow-iclr/recflow","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"CC-BY-SA-4.0"}}],"tasks":[{"task_slug":"recommendation-systems","task_name":"Recommendation Systems"},{"task_slug":"selection-bias","task_name":"Selection bias"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2410.20868","atlas_url":"https://app.syntology.ai/?focus=2410.20868","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.20868"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/recflow-iclr/recflow","reach":{"status":"ok","spdx":"CC-BY-SA-4.0"}}],"summary":{"unverified":2},"by_repo_kind":{"official":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"0e11171ee3a742b5","entry":"evaluate","repo":"recflow-iclr/recflow","repo_kind":"official","path":"coarse/metrics.py","file_url":"https://github.com/recflow-iclr/recflow/blob/HEAD/coarse/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"0e11171ee3a742b5"}},{"code_sha256_prefix":"a07d85e6ce14e3bc","entry":"evaluate_recall","repo":"recflow-iclr/recflow","repo_kind":"official","path":"coarse/metrics.py","file_url":"https://github.com/recflow-iclr/recflow/blob/HEAD/coarse/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"a07d85e6ce14e3bc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}