{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2603-11008","title":"A Systematic Study of Pseudo-Relevance Feedback with LLMs","arxiv_id":"2603.11008","date":"2026-03-11","proceeding":null,"authors":["Nour Jedidi","Jimmy Lin"],"abstract":"Pseudo-relevance feedback (PRF) methods built on large language models (LLMs) can be organized along two key design dimensions: the feedback source, which is where the feedback text is derived from and the feedback model, which is how the given feedback text is used to refine the query representation. However, the independent role that each dimension plays is unclear, as both are often entangled in empirical evaluations. In this paper, we address this gap by systematically studying how the choice of feedback source and feedback model impact PRF effectiveness through controlled experimentation. Across 13 low-resource BEIR tasks with five LLM PRF methods, our results show: (1) the choice of feedback model can play a critical role in PRF effectiveness; (2) feedback derived solely from LLM-generated text provides the most cost-effective solution; and (3) feedback derived from the corpus is most beneficial when utilizing candidate documents from a strong first-stage retriever. Together, our findings provide a better understanding of which elements in the PRF design space are most important.","url_abs":"https://arxiv.org/abs/2603.11008","url_pdf":"https://arxiv.org/pdf/2603.11008","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2603.11008","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2603.11008"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/castorini/llm-relevance-feedback","reach":null}],"summary":{"unverified":8},"by_repo_kind":{"found_in_text":{"samples":8,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":8,"samples":[{"code_sha256_prefix":"b28d0bd833bcdaa7","entry":"generate_dense_hyde_query_from_feedback","repo":"castorini/llm-relevance-feedback","repo_kind":"found_in_text","path":"run_hyde.py","file_url":"https://github.com/castorini/llm-relevance-feedback/blob/HEAD/run_hyde.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b28d0bd833bcdaa7"}},{"code_sha256_prefix":"711e41e4de979199","entry":"generate_dense_hyde_query_from_feedback","repo":"castorini/llm-relevance-feedback","repo_kind":"found_in_text","path":"run_hyde_prf.py","file_url":"https://github.com/castorini/llm-relevance-feedback/blob/HEAD/run_hyde_prf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"711e41e4de979199"}},{"code_sha256_prefix":"389835acaf3c7981","entry":"get_document_vector_rm3","repo":"castorini/llm-relevance-feedback","repo_kind":"found_in_text","path":"feedback_methods/rm3_feedback.py","file_url":"https://github.com/castorini/llm-relevance-feedback/blob/HEAD/feedback_methods/rm3_feedback.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"389835acaf3c7981"}},{"code_sha256_prefix":"d5a3b84495aeff2e","entry":"get_document_vector_rocchio","repo":"castorini/llm-relevance-feedback","repo_kind":"found_in_text","path":"feedback_methods/rocchio_feedback.py","file_url":"https://github.com/castorini/llm-relevance-feedback/blob/HEAD/feedback_methods/rocchio_feedback.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d5a3b84495aeff2e"}},{"code_sha256_prefix":"ad315a0a59dbff91","entry":"l1_normalize","repo":"castorini/llm-relevance-feedback","repo_kind":"found_in_text","path":"feedback_methods/rm3_feedback.py","file_url":"https://github.com/castorini/llm-relevance-feedback/blob/HEAD/feedback_methods/rm3_feedback.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ad315a0a59dbff91"}},{"code_sha256_prefix":"6b5ae88f73a2c185","entry":"load_pickle_if_available","repo":"castorini/llm-relevance-feedback","repo_kind":"found_in_text","path":"modules/helpers.py","file_url":"https://github.com/castorini/llm-relevance-feedback/blob/HEAD/modules/helpers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6b5ae88f73a2c185"}},{"code_sha256_prefix":"793056582eb7e42c","entry":"prune_top_k","repo":"castorini/llm-relevance-feedback","repo_kind":"found_in_text","path":"feedback_methods/rm3_feedback.py","file_url":"https://github.com/castorini/llm-relevance-feedback/blob/HEAD/feedback_methods/rm3_feedback.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"793056582eb7e42c"}},{"code_sha256_prefix":"148e3c68d6c372ac","entry":"run_bm25","repo":"castorini/llm-relevance-feedback","repo_kind":"found_in_text","path":"modules/helpers.py","file_url":"https://github.com/castorini/llm-relevance-feedback/blob/HEAD/modules/helpers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"148e3c68d6c372ac"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}