{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/instruction-distillation-makes-large-language","title":"Instruction Distillation Makes Large Language Models Efficient Zero-shot Rankers","arxiv_id":"2311.01555","date":"2023-11-02","proceeding":null,"authors":["Weiwei Sun","Zheng Chen","Xinyu Ma","Lingyong Yan","Shuaiqiang Wang","Pengjie Ren","Zhumin Chen","Dawei Yin","Zhaochun Ren"],"abstract":"Recent studies have demonstrated the great potential of Large Language Models (LLMs) serving as zero-shot relevance rankers. The typical approach involves making comparisons between pairs or lists of documents. Although effective, these listwise and pairwise methods are not efficient and also heavily rely on intricate prompt engineering. To tackle this problem, we introduce a novel instruction distillation method. The key idea is to distill the pairwise ranking ability of open-sourced LLMs to a simpler but more efficient pointwise ranking. Specifically, given the same LLM, we first rank documents using the effective pairwise approach with complex instructions, and then distill the teacher predictions to the pointwise approach with simpler instructions. Evaluation results on the BEIR, TREC, and ReDial datasets demonstrate that instruction distillation can improve efficiency by 10 to 100x and also enhance the ranking performance of LLMs. Furthermore, our approach surpasses the performance of existing supervised methods like monoT5 and is on par with the state-of-the-art zero-shot methods. The code to reproduce our results is available at www.github.com/sunnweiwei/RankGPT.","url_abs":"https://arxiv.org/abs/2311.01555v1","url_pdf":"https://arxiv.org/pdf/2311.01555v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"instruction-distillation-makes-large-language","repo_url":"https://github.com/sunnweiwei/rankgpt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"prompt-engineering","task_name":"Prompt Engineering"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2311.01555","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2311.01555"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sunnweiwei/rankgpt","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":11,"unverified":4},"by_repo_kind":{"official":{"samples":15,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8e9717b5010ed9a8","entry":"convert_messages_to_prompt","repo":"sunnweiwei/rankgpt","repo_kind":"official","path":"rank_gpt.py","file_url":"https://github.com/sunnweiwei/rankgpt/blob/HEAD/rank_gpt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8e9717b5010ed9a8"}},{"code_sha256_prefix":"1eb76fdb17d065e3","entry":"gather_tensors","repo":"sunnweiwei/rankgpt","repo_kind":"official","path":"InstructDistill/instruction_distill.py","file_url":"https://github.com/sunnweiwei/rankgpt/blob/HEAD/InstructDistill/instruction_distill.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1eb76fdb17d065e3"}},{"code_sha256_prefix":"64789829d440d8a9","entry":"get_prefix_prompt","repo":"sunnweiwei/rankgpt","repo_kind":"official","path":"rank_gpt.py","file_url":"https://github.com/sunnweiwei/rankgpt/blob/HEAD/rank_gpt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"64789829d440d8a9"}},{"code_sha256_prefix":"5160f8fbdd1c6606","entry":"get_qrels_file","repo":"sunnweiwei/rankgpt","repo_kind":"official","path":"InstructDistill/trec_eval.py","file_url":"https://github.com/sunnweiwei/rankgpt/blob/HEAD/InstructDistill/trec_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5160f8fbdd1c6606"}},{"code_sha256_prefix":"9c71991dafef6c40","entry":"lambdaRank_scheme","repo":"sunnweiwei/rankgpt","repo_kind":"official","path":"rank_loss.py","file_url":"https://github.com/sunnweiwei/rankgpt/blob/HEAD/rank_loss.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9c71991dafef6c40"}},{"code_sha256_prefix":"07141353fded7e46","entry":"ndcgLoss1_scheme","repo":"sunnweiwei/rankgpt","repo_kind":"official","path":"rank_loss.py","file_url":"https://github.com/sunnweiwei/rankgpt/blob/HEAD/rank_loss.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"07141353fded7e46"}},{"code_sha256_prefix":"103f50dcc89657c1","entry":"receive_response","repo":"sunnweiwei/rankgpt","repo_kind":"official","path":"specialization.py","file_url":"https://github.com/sunnweiwei/rankgpt/blob/HEAD/specialization.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"103f50dcc89657c1"}},{"code_sha256_prefix":"4fc539f7640d8633","entry":"remove_duplicate","repo":"sunnweiwei/rankgpt","repo_kind":"official","path":"trec_eval.py","file_url":"https://github.com/sunnweiwei/rankgpt/blob/HEAD/trec_eval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4fc539f7640d8633"}},{"code_sha256_prefix":"638bf99dbd386233","entry":"run_retriever","repo":"sunnweiwei/rankgpt","repo_kind":"official","path":"rank_gpt.py","file_url":"https://github.com/sunnweiwei/rankgpt/blob/HEAD/rank_gpt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"638bf99dbd386233"}},{"code_sha256_prefix":"43e84c75d62b7d6c","entry":"run_retriever","repo":"sunnweiwei/rankgpt","repo_kind":"official","path":"InstructDistill/bm25_retrieval.py","file_url":"https://github.com/sunnweiwei/rankgpt/blob/HEAD/InstructDistill/bm25_retrieval.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"43e84c75d62b7d6c"}},{"code_sha256_prefix":"654ad1edd7372dd6","entry":"split_data","repo":"sunnweiwei/rankgpt","repo_kind":"official","path":"InstructDistill/instruction_distill.py","file_url":"https://github.com/sunnweiwei/rankgpt/blob/HEAD/InstructDistill/instruction_distill.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"654ad1edd7372dd6"}},{"code_sha256_prefix":"ce1221b963c84b7c","entry":"get_qrels_file","repo":"sunnweiwei/rankgpt","repo_kind":"official","path":"trec_eval.py","file_url":"https://github.com/sunnweiwei/rankgpt/blob/HEAD/trec_eval.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ce1221b963c84b7c"}},{"code_sha256_prefix":"c1fb53abb507ff8e","entry":"ndcgLoss2_scheme","repo":"sunnweiwei/rankgpt","repo_kind":"official","path":"rank_loss.py","file_url":"https://github.com/sunnweiwei/rankgpt/blob/HEAD/rank_loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c1fb53abb507ff8e"}},{"code_sha256_prefix":"a6d651ee8a87c18b","entry":"query_generation","repo":"sunnweiwei/rankgpt","repo_kind":"official","path":"pointwise.py","file_url":"https://github.com/sunnweiwei/rankgpt/blob/HEAD/pointwise.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a6d651ee8a87c18b"}},{"code_sha256_prefix":"7586cb0204e57b53","entry":"relevance_generation","repo":"sunnweiwei/rankgpt","repo_kind":"official","path":"pointwise.py","file_url":"https://github.com/sunnweiwei/rankgpt/blob/HEAD/pointwise.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7586cb0204e57b53"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}