{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/chain-of-knowledge-a-framework-for-grounding","title":"Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sources","arxiv_id":"2305.13269","date":"2023-05-22","proceeding":null,"authors":["Xingxuan Li","Ruochen Zhao","Yew Ken Chia","Bosheng Ding","Shafiq Joty","Soujanya Poria","Lidong Bing"],"abstract":"We present chain-of-knowledge (CoK), a novel framework that augments large language models (LLMs) by dynamically incorporating grounding information from heterogeneous sources. It results in more factual rationales and reduced hallucination in generation. Specifically, CoK consists of three stages: reasoning preparation, dynamic knowledge adapting, and answer consolidation. Given a knowledge-intensive question, CoK first prepares several preliminary rationales and answers while identifying the relevant knowledge domains. If there is no majority consensus among the answers from samples, CoK corrects the rationales step by step by adapting knowledge from the identified domains. These corrected rationales can plausibly serve as a better foundation for the final answer consolidation. Unlike prior studies that primarily use unstructured data, CoK also leverages structured knowledge sources such as Wikidata and tables that provide more reliable factual information. To access both unstructured and structured knowledge sources in the dynamic knowledge adapting stage, we propose an adaptive query generator that allows the generation of queries for various types of query languages, including SPARQL, SQL, and natural sentences. Moreover, to minimize error propagation between rationales, CoK corrects the rationales progressively using preceding corrected rationales to generate and correct subsequent rationales. Extensive experiments show that CoK consistently improves the performance of LLMs on knowledge-intensive tasks across different domains.","url_abs":"https://arxiv.org/abs/2305.13269v4","url_pdf":"https://arxiv.org/pdf/2305.13269v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"chain-of-knowledge-a-framework-for-grounding","repo_url":"https://github.com/damo-nlp-sg/chain-of-knowledge","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"hallucination","task_name":"Hallucination"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"large-language-model","task_name":"Large Language Model"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.13269","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.13269"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/DAMO-NLP-SG/chain-of-knowledge","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/damo-nlp-sg/chain-of-knowledge","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_violates":1,"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":1,"repositories":1},"community":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"dd4646b11bae8744","entry":"knowl_is_empty","repo":"DAMO-NLP-SG/chain-of-knowledge","repo_kind":"official","path":"utils/knowl_query.py","file_url":"https://github.com/DAMO-NLP-SG/chain-of-knowledge/blob/HEAD/utils/knowl_query.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dd4646b11bae8744"}},{"code_sha256_prefix":"e1cb843788df2e0e","entry":"call_openai_api","repo":"damo-nlp-sg/chain-of-knowledge","repo_kind":"official","path":"utils/openai_utils.py","file_url":"https://github.com/damo-nlp-sg/chain-of-knowledge/blob/HEAD/utils/openai_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"e1cb843788df2e0e"}},{"code_sha256_prefix":"568afe28168c3da5","entry":"formatting_prompts_func","repo":"damo-nlp-sg/chain-of-knowledge","repo_kind":"official","path":"finetune/sft_trainer.py","file_url":"https://github.com/damo-nlp-sg/chain-of-knowledge/blob/HEAD/finetune/sft_trainer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"568afe28168c3da5"}},{"code_sha256_prefix":"28145bbcf1d4f9ea","entry":"print_now","repo":"wjn1996/Chain-of-Knowledge","repo_kind":"community","path":"framework/model.py","file_url":"https://github.com/wjn1996/Chain-of-Knowledge/blob/HEAD/framework/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"28145bbcf1d4f9ea"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}