{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/list-aware-reranking-truncation-joint-model","title":"List-aware Reranking-Truncation Joint Model for Search and Retrieval-augmented Generation","arxiv_id":"2402.02764","date":"2024-02-05","proceeding":null,"authors":["Shicheng Xu","Liang Pang","Jun Xu","HuaWei Shen","Xueqi Cheng"],"abstract":"The results of information retrieval (IR) are usually presented in the form of a ranked list of candidate documents, such as web search for humans and retrieval-augmented generation for large language models (LLMs). List-aware retrieval aims to capture the list-level contextual features to return a better list, mainly including reranking and truncation. Reranking finely re-scores the documents in the list. Truncation dynamically determines the cut-off point of the ranked list to achieve the trade-off between overall relevance and avoiding misinformation from irrelevant documents. Previous studies treat them as two separate tasks and model them separately. However, the separation is not optimal. First, it is hard to share the contextual information of the ranking list between the two tasks. Second, the separate pipeline usually meets the error accumulation problem, where the small error from the reranking stage can largely affect the truncation stage. To solve these problems, we propose a Reranking-Truncation joint model (GenRT) that can perform the two tasks concurrently. GenRT integrates reranking and truncation via generative paradigm based on encoder-decoder architecture. We also design the novel loss functions for joint optimization to make the model learn both tasks. Sharing parameters by the joint model is conducive to making full use of the common modeling information of the two tasks. Besides, the two tasks are performed concurrently and co-optimized to solve the error accumulation problem between separate stages. Experiments on public learning-to-rank benchmarks and open-domain Q\\&A tasks show that our method achieves SOTA performance on both reranking and truncation tasks for web search and retrieval-augmented LLMs.","url_abs":"https://arxiv.org/abs/2402.02764v1","url_pdf":"https://arxiv.org/pdf/2402.02764v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"list-aware-reranking-truncation-joint-model","repo_url":"https://github.com/xsc1234/genrt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"jax","reach":{"status":"ok"}}],"tasks":[{"task_slug":"information-retrieval","task_name":"Information Retrieval"},{"task_slug":"learning-to-rank","task_name":"Learning-To-Rank"},{"task_slug":"misinformation","task_name":"Misinformation"},{"task_slug":"reranking","task_name":"Reranking"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"retrieval-augmented-generation","task_name":"Retrieval-augmented Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2402.02764","atlas_url":"https://app.syntology.ai/?focus=2402.02764","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.02764"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xsc1234/genrt","reach":{"status":"ok"}}],"summary":{"ran":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"a3a7ab754fcdace0","entry":"get_dcg","repo":"xsc1234/genrt","repo_kind":"official","path":"model/modeling_t5_add_pos_eof.py","file_url":"https://github.com/xsc1234/genrt/blob/HEAD/model/modeling_t5_add_pos_eof.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a3a7ab754fcdace0"}},{"code_sha256_prefix":"7656fe08e8e1f335","entry":"get_ndcg","repo":"xsc1234/genrt","repo_kind":"official","path":"model/modeling_t5_add_pos_eof.py","file_url":"https://github.com/xsc1234/genrt/blob/HEAD/model/modeling_t5_add_pos_eof.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7656fe08e8e1f335"}},{"code_sha256_prefix":"653e646630924a5c","entry":"get_truncate_dcg","repo":"xsc1234/genrt","repo_kind":"official","path":"model/modeling_t5_add_pos_eof.py","file_url":"https://github.com/xsc1234/genrt/blob/HEAD/model/modeling_t5_add_pos_eof.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"653e646630924a5c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}