{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/small-models-big-insights-leveraging-slim","title":"Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs","arxiv_id":"2402.12052","date":"2024-02-19","proceeding":null,"authors":["Jiejun Tan","Zhicheng Dou","Yutao Zhu","Peidong Guo","Kun Fang","Ji-Rong Wen"],"abstract":"The integration of large language models (LLMs) and search engines represents a significant evolution in knowledge acquisition methodologies. However, determining the knowledge that an LLM already possesses and the knowledge that requires the help of a search engine remains an unresolved issue. Most existing methods solve this problem through the results of preliminary answers or reasoning done by the LLM itself, but this incurs excessively high computational costs. This paper introduces a novel collaborative approach, namely SlimPLM, that detects missing knowledge in LLMs with a slim proxy model, to enhance the LLM's knowledge acquisition process. We employ a proxy model which has far fewer parameters, and take its answers as heuristic answers. Heuristic answers are then utilized to predict the knowledge required to answer the user question, as well as the known and unknown knowledge within the LLM. We only conduct retrieval for the missing knowledge in questions that the LLM does not know. Extensive experimental results on five datasets with two LLMs demonstrate a notable improvement in the end-to-end performance of LLMs in question-answering tasks, achieving or surpassing current state-of-the-art models with lower LLM inference costs.","url_abs":"https://arxiv.org/abs/2402.12052v3","url_pdf":"https://arxiv.org/pdf/2402.12052v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"small-models-big-insights-leveraging-slim","repo_url":"https://github.com/plageon/slimplm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.12052","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.12052"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/plageon/slimplm","reach":{"status":"ok"}}],"summary":{"ran":5,"ran_draft_wrong":1,"unverified":9},"by_repo_kind":{"official":{"samples":15,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":15,"samples":[{"code_sha256_prefix":"0be284da9bf9ca86","entry":"exact_match_score","repo":"plageon/slimplm","repo_kind":"official","path":"SKR/skr.py","file_url":"https://github.com/plageon/slimplm/blob/HEAD/SKR/skr.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0be284da9bf9ca86"}},{"code_sha256_prefix":"42ae926df6227a65","entry":"f1_score","repo":"plageon/slimplm","repo_kind":"official","path":"SKR/skr.py","file_url":"https://github.com/plageon/slimplm/blob/HEAD/SKR/skr.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"42ae926df6227a65"}},{"code_sha256_prefix":"1616f54ad713d861","entry":"get_stop_words_ids","repo":"plageon/slimplm","repo_kind":"official","path":"SKR/vllm_wrapper.py","file_url":"https://github.com/plageon/slimplm/blob/HEAD/SKR/vllm_wrapper.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1616f54ad713d861"}},{"code_sha256_prefix":"cc910a23dfa4faf9","entry":"make_context","repo":"plageon/slimplm","repo_kind":"official","path":"SKR/vllm_wrapper.py","file_url":"https://github.com/plageon/slimplm/blob/HEAD/SKR/vllm_wrapper.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"cc910a23dfa4faf9"}},{"code_sha256_prefix":"ae914f80513baf0f","entry":"normalize_answer","repo":"plageon/slimplm","repo_kind":"official","path":"SKR/skr.py","file_url":"https://github.com/plageon/slimplm/blob/HEAD/SKR/skr.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ae914f80513baf0f"}},{"code_sha256_prefix":"4559d324e8869534","entry":"parse_query_rewrite_result","repo":"plageon/slimplm","repo_kind":"official","path":"itergen/itergen_search.py","file_url":"https://github.com/plageon/slimplm/blob/HEAD/itergen/itergen_search.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4559d324e8869534"}},{"code_sha256_prefix":"0d1849e93ded0365","entry":"apply_retrieval","repo":"plageon/slimplm","repo_kind":"official","path":"itergen/itergen_search.py","file_url":"https://github.com/plageon/slimplm/blob/HEAD/itergen/itergen_search.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0d1849e93ded0365"}},{"code_sha256_prefix":"24880d8347224f43","entry":"cal_logits","repo":"plageon/slimplm","repo_kind":"official","path":"flare/flare_search.py","file_url":"https://github.com/plageon/slimplm/blob/HEAD/flare/flare_search.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"24880d8347224f43"}},{"code_sha256_prefix":"b15ebcca575175b2","entry":"chat_prompt_search","repo":"plageon/slimplm","repo_kind":"official","path":"itergen/itergen_chat.py","file_url":"https://github.com/plageon/slimplm/blob/HEAD/itergen/itergen_chat.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b15ebcca575175b2"}},{"code_sha256_prefix":"9264b80c05ab8367","entry":"check_chat_result","repo":"plageon/slimplm","repo_kind":"official","path":"flare/flare_search.py","file_url":"https://github.com/plageon/slimplm/blob/HEAD/flare/flare_search.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9264b80c05ab8367"}},{"code_sha256_prefix":"017881a697f8102f","entry":"first_round_inference_req","repo":"plageon/slimplm","repo_kind":"official","path":"flare/flare_chat_inference.py","file_url":"https://github.com/plageon/slimplm/blob/HEAD/flare/flare_chat_inference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"017881a697f8102f"}},{"code_sha256_prefix":"a49b2e989e8da871","entry":"get_dataset","repo":"plageon/slimplm","repo_kind":"official","path":"user_intent/finetune_llm.py","file_url":"https://github.com/plageon/slimplm/blob/HEAD/user_intent/finetune_llm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a49b2e989e8da871"}},{"code_sha256_prefix":"e6dea7666ba5f046","entry":"sending_inference_request","repo":"plageon/slimplm","repo_kind":"official","path":"SKR/icl.py","file_url":"https://github.com/plageon/slimplm/blob/HEAD/SKR/icl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e6dea7666ba5f046"}},{"code_sha256_prefix":"41b5dc31cd322ee3","entry":"sending_inference_request","repo":"plageon/slimplm","repo_kind":"official","path":"cot/cot_prompt_inference.py","file_url":"https://github.com/plageon/slimplm/blob/HEAD/cot/cot_prompt_inference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"41b5dc31cd322ee3"}},{"code_sha256_prefix":"b3e3436d21b2bc92","entry":"sending_rewrite_request","repo":"plageon/slimplm","repo_kind":"official","path":"itergen/itergen_search.py","file_url":"https://github.com/plageon/slimplm/blob/HEAD/itergen/itergen_search.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b3e3436d21b2bc92"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}