{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/think-on-graph-2-0-deep-and-interpretable","title":"Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generation","arxiv_id":"2407.10805","date":"2024-07-15","proceeding":null,"authors":["Shengjie Ma","Chengjin Xu","Xuhui Jiang","Muzhi Li","Huaren Qu","Cehao Yang","Jiaxin Mao","Jian Guo"],"abstract":"Retrieval-augmented generation (RAG) has improved large language models (LLMs) by using knowledge retrieval to overcome knowledge deficiencies. However, current RAG methods often fall short of ensuring the depth and completeness of retrieved information, which is necessary for complex reasoning tasks. In this work, we introduce Think-on-Graph 2.0 (ToG-2), a hybrid RAG framework that iteratively retrieves information from both unstructured and structured knowledge sources in a tight-coupling manner. Specifically, ToG-2 leverages knowledge graphs (KGs) to link documents via entities, facilitating deep and knowledge-guided context retrieval. Simultaneously, it utilizes documents as entity contexts to achieve precise and efficient graph retrieval. ToG-2 alternates between graph retrieval and context retrieval to search for in-depth clues relevant to the question, enabling LLMs to generate answers. We conduct a series of well-designed experiments to highlight the following advantages of ToG-2: 1) ToG-2 tightly couples the processes of context retrieval and graph retrieval, deepening context retrieval via the KG while enabling reliable graph retrieval based on contexts; 2) it achieves deep and faithful reasoning in LLMs through an iterative knowledge retrieval process of collaboration between contexts and the KG; and 3) ToG-2 is training-free and plug-and-play compatible with various LLMs. Extensive experiments demonstrate that ToG-2 achieves overall state-of-the-art (SOTA) performance on 6 out of 7 knowledge-intensive datasets with GPT-3.5, and can elevate the performance of smaller models (e.g., LLAMA-2-13B) to the level of GPT-3.5's direct reasoning. The source code is available on https://github.com/IDEA-FinAI/ToG-2.","url_abs":"https://arxiv.org/abs/2407.10805v6","url_pdf":"https://arxiv.org/pdf/2407.10805v6.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"think-on-graph-2-0-deep-and-interpretable","repo_url":"https://github.com/idea-finai/tog-2","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"information-retrieval","task_name":"Information Retrieval"},{"task_slug":"knowledge-graphs","task_name":"Knowledge Graphs"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"large-language-model","task_name":"Large Language Model"},{"task_slug":"rag","task_name":"RAG"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"retrieval-augmented-generation","task_name":"Retrieval-augmented Generation"},{"task_slug":"semantic-similarity","task_name":"Semantic Similarity"},{"task_slug":"semantic-textual-similarity","task_name":"Semantic Textual Similarity"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"bart","method_name":"BART"},{"method_slug":"bert","method_name":"BERT"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"cosine-annealing","method_name":"Cosine Annealing"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt-3","method_name":"GPT-3"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-cosine-annealing","method_name":"Linear Warmup With Cosine Annealing"},{"method_slug":"linear-warmup-with-linear-decay","method_name":"Linear Warmup With Linear Decay"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"rag","method_name":"RAG"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"weight-decay","method_name":"Weight Decay"},{"method_slug":"wordpiece","method_name":"WordPiece"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.10805","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.10805"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/idea-finai/tog-2","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":12,"ran_honours":1,"unverified":3},"by_repo_kind":{"official":{"samples":16,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d858076ab384807e","entry":"align","repo":"idea-finai/tog-2","repo_kind":"official","path":"eval/utils.py","file_url":"https://github.com/idea-finai/tog-2/blob/HEAD/eval/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d858076ab384807e"}},{"code_sha256_prefix":"9ceaedf109fba536","entry":"biencoder_similarity","repo":"idea-finai/tog-2","repo_kind":"official","path":"ToG-2/search.py","file_url":"https://github.com/idea-finai/tog-2/blob/HEAD/ToG-2/search.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9ceaedf109fba536"}},{"code_sha256_prefix":"d7a01348958713cb","entry":"check_string","repo":"idea-finai/tog-2","repo_kind":"official","path":"eval/utils.py","file_url":"https://github.com/idea-finai/tog-2/blob/HEAD/eval/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d7a01348958713cb"}},{"code_sha256_prefix":"1d7cb1f72b14a5d4","entry":"compute_bm25_similarity","repo":"idea-finai/tog-2","repo_kind":"official","path":"ToG-2/utils.py","file_url":"https://github.com/idea-finai/tog-2/blob/HEAD/ToG-2/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1d7cb1f72b14a5d4"}},{"code_sha256_prefix":"6e4a42e284551029","entry":"crossencoder_similarity","repo":"idea-finai/tog-2","repo_kind":"official","path":"ToG-2/search.py","file_url":"https://github.com/idea-finai/tog-2/blob/HEAD/ToG-2/search.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6e4a42e284551029"}},{"code_sha256_prefix":"1fba9a1516cbfeed","entry":"entity_linking_azure_cache","repo":"idea-finai/tog-2","repo_kind":"official","path":"ToG-2/ner.py","file_url":"https://github.com/idea-finai/tog-2/blob/HEAD/ToG-2/ner.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1fba9a1516cbfeed"}},{"code_sha256_prefix":"56587c771bc3250d","entry":"entity_linking_azure_search","repo":"idea-finai/tog-2","repo_kind":"official","path":"ToG-2/ner.py","file_url":"https://github.com/idea-finai/tog-2/blob/HEAD/ToG-2/ner.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"56587c771bc3250d"}},{"code_sha256_prefix":"3d56eaa0b0bfd370","entry":"format_entity_name_for_wikipedia","repo":"idea-finai/tog-2","repo_kind":"official","path":"ToG-2/client.py","file_url":"https://github.com/idea-finai/tog-2/blob/HEAD/ToG-2/client.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3d56eaa0b0bfd370"}},{"code_sha256_prefix":"54c796e1cf61d9f9","entry":"get_batch_files","repo":"idea-finai/tog-2","repo_kind":"official","path":"Wikidata/simple_wikidata_db/utils.py","file_url":"https://github.com/idea-finai/tog-2/blob/HEAD/Wikidata/simple_wikidata_db/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"54c796e1cf61d9f9"}},{"code_sha256_prefix":"d29ad9e91ad3ef23","entry":"if_all_zero","repo":"idea-finai/tog-2","repo_kind":"official","path":"ToG-2/utils.py","file_url":"https://github.com/idea-finai/tog-2/blob/HEAD/ToG-2/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d29ad9e91ad3ef23"}},{"code_sha256_prefix":"989b2db8e3d1abc0","entry":"scores_rank","repo":"idea-finai/tog-2","repo_kind":"official","path":"ToG-2/search.py","file_url":"https://github.com/idea-finai/tog-2/blob/HEAD/ToG-2/search.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"989b2db8e3d1abc0"}},{"code_sha256_prefix":"db87f61e2e22ab6f","entry":"sliding_window_type","repo":"idea-finai/tog-2","repo_kind":"official","path":"ToG-2/main_tog2.py","file_url":"https://github.com/idea-finai/tog-2/blob/HEAD/ToG-2/main_tog2.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"db87f61e2e22ab6f"}},{"code_sha256_prefix":"d1a2d46886246d95","entry":"transform_relation","repo":"idea-finai/tog-2","repo_kind":"official","path":"ToG-2/wiki_func.py","file_url":"https://github.com/idea-finai/tog-2/blob/HEAD/ToG-2/wiki_func.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d1a2d46886246d95"}},{"code_sha256_prefix":"962af27b3c6f4838","entry":"clean_relation_all_e","repo":"idea-finai/tog-2","repo_kind":"official","path":"ToG-2/wiki_func.py","file_url":"https://github.com/idea-finai/tog-2/blob/HEAD/ToG-2/wiki_func.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"962af27b3c6f4838"}},{"code_sha256_prefix":"b93bb663c55da34d","entry":"clean_relations","repo":"idea-finai/tog-2","repo_kind":"official","path":"ToG-2/wiki_func.py","file_url":"https://github.com/idea-finai/tog-2/blob/HEAD/ToG-2/wiki_func.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b93bb663c55da34d"}},{"code_sha256_prefix":"f0c6d1dc03117242","entry":"prepare_dataset_for_eval","repo":"idea-finai/tog-2","repo_kind":"official","path":"eval/utils.py","file_url":"https://github.com/idea-finai/tog-2/blob/HEAD/eval/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f0c6d1dc03117242"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}