{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/paths-over-graph-knowledge-graph-enpowered","title":"Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoning","arxiv_id":"2410.14211","date":"2024-10-18","proceeding":null,"authors":["Xingyu Tan","Xiaoyang Wang","Qing Liu","Xiwei Xu","Xin Yuan","Wenjie Zhang"],"abstract":"Large Language Models (LLMs) have achieved impressive results in various tasks but struggle with hallucination problems and lack of relevant knowledge, especially in deep complex reasoning and knowledge-intensive tasks. Knowledge Graphs (KGs), which capture vast amounts of facts in a structured format, offer a reliable source of knowledge for reasoning. However, existing KG-based LLM reasoning methods face challenges like handling multi-hop reasoning, multi-entity questions, and effectively utilizing graph structures. To address these issues, we propose Paths-over-Graph (PoG), a novel method that enhances LLM reasoning by integrating knowledge reasoning paths from KGs, improving the interpretability and faithfulness of LLM outputs. PoG tackles multi-hop and multi-entity questions through a three-phase dynamic multi-hop path exploration, which combines the inherent knowledge of LLMs with factual knowledge from KGs. In order to improve the efficiency, PoG prunes irrelevant information from the graph exploration first and introduces efficient three-step pruning techniques that incorporate graph structures, LLM prompting, and a pre-trained language model (e.g., SBERT) to effectively narrow down the explored candidate paths. This ensures all reasoning paths contain highly relevant information captured from KGs, making the reasoning faithful and interpretable in problem-solving. PoG innovatively utilizes graph structure to prune the irrelevant noise and represents the first method to implement multi-entity deep path detection on KGs for LLM reasoning tasks. Comprehensive experiments on five benchmark KGQA datasets demonstrate PoG outperforms the state-of-the-art method ToG across GPT-3.5-Turbo and GPT-4, achieving an average accuracy improvement of 18.9%. Notably, PoG with GPT-3.5-Turbo surpasses ToG with GPT-4 by up to 23.9%.","url_abs":"https://arxiv.org/abs/2410.14211v3","url_pdf":"https://arxiv.org/pdf/2410.14211v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"paths-over-graph-knowledge-graph-enpowered","repo_url":"https://github.com/SteveTANTAN/PoG","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"hallucination","task_name":"Hallucination"},{"task_slug":"knowledge-base-question-answering","task_name":"Knowledge Base Question Answering"},{"task_slug":"knowledge-graphs","task_name":"Knowledge Graphs"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"large-language-model","task_name":"Large Language Model"},{"task_slug":"question-answering","task_name":"Question Answering"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"cosine-annealing","method_name":"Cosine Annealing"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt-3","method_name":"GPT-3"},{"method_slug":"gpt-4","method_name":"GPT-4"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-cosine-annealing","method_name":"Linear Warmup With Cosine Annealing"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"pruning","method_name":"Pruning"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"},{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/knowledge-base-question-answering-on","task":"Knowledge Base Question Answering","dataset":"ComplexWebQuestions","model":"PoG-GPT4 (Tan et al., 2024)","rank_in_archive_order":6,"of":6,"metrics":{"EM":"81.4"},"uses_additional_data":false},{"leaderboard":"/sota/knowledge-base-question-answering-on-grailqa","task":"Knowledge Base Question Answering","dataset":"GrailQA","model":"PoG-GPT4 (Tan et al., 2024)","rank_in_archive_order":2,"of":2,"metrics":{"EM":"94.4"},"uses_additional_data":false},{"leaderboard":"/sota/knowledge-base-question-answering-on-2","task":"Knowledge Base Question Answering","dataset":"SimpleQuestions","model":"PoG-GPT4 (Tan et al., 2024)","rank_in_archive_order":1,"of":1,"metrics":{"EM":"84"},"uses_additional_data":false},{"leaderboard":"/sota/knowledge-base-question-answering-on-webqsp-1","task":"Knowledge Base Question Answering","dataset":"WebQSP","model":"PoG-GPT4 (Tan et al., 2024)","rank_in_archive_order":1,"of":1,"metrics":{"EM":"96.7"},"uses_additional_data":false},{"leaderboard":"/sota/knowledge-base-question-answering-on-3","task":"Knowledge Base Question Answering","dataset":"WebQuestions","model":"PoG-GPT4 (Tan et al., 2024)","rank_in_archive_order":1,"of":1,"metrics":{"EM":"84.6"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.14211","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.14211"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/SteveTANTAN/PoG","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":1,"unverified":11},"by_repo_kind":{"listed":{"samples":12,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ffbc2c4b715e9ad2","entry":"check_end_word","repo":"SteveTANTAN/PoG","repo_kind":"listed","path":"PoG/freebase_func.py","file_url":"https://github.com/SteveTANTAN/PoG/blob/HEAD/PoG/freebase_func.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ffbc2c4b715e9ad2"}},{"code_sha256_prefix":"8efd8b00e5ee004a","entry":"abandon_rels","repo":"SteveTANTAN/PoG","repo_kind":"listed","path":"PoG/freebase_func.py","file_url":"https://github.com/SteveTANTAN/PoG/blob/HEAD/PoG/freebase_func.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8efd8b00e5ee004a"}},{"code_sha256_prefix":"d1b8a0d48a7da6f9","entry":"check_in_path","repo":"SteveTANTAN/PoG","repo_kind":"listed","path":"PoG/check_answer.py","file_url":"https://github.com/SteveTANTAN/PoG/blob/HEAD/PoG/check_answer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d1b8a0d48a7da6f9"}},{"code_sha256_prefix":"dcb9359b01d13cbe","entry":"count_tokens","repo":"SteveTANTAN/PoG","repo_kind":"listed","path":"PoG/utils.py","file_url":"https://github.com/SteveTANTAN/PoG/blob/HEAD/PoG/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dcb9359b01d13cbe"}},{"code_sha256_prefix":"f04904e3d891cf41","entry":"entity_need_explore","repo":"SteveTANTAN/PoG","repo_kind":"listed","path":"PoG/subgraph_utilts.py","file_url":"https://github.com/SteveTANTAN/PoG/blob/HEAD/PoG/subgraph_utilts.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f04904e3d891cf41"}},{"code_sha256_prefix":"76234c3134b81ef7","entry":"extract_entities","repo":"SteveTANTAN/PoG","repo_kind":"listed","path":"PoG/PoG_multi.py","file_url":"https://github.com/SteveTANTAN/PoG/blob/HEAD/PoG/PoG_multi.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"76234c3134b81ef7"}},{"code_sha256_prefix":"6fb549457e334782","entry":"extract_top_list","repo":"SteveTANTAN/PoG","repo_kind":"listed","path":"PoG/PoG_multi.py","file_url":"https://github.com/SteveTANTAN/PoG/blob/HEAD/PoG/PoG_multi.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6fb549457e334782"}},{"code_sha256_prefix":"4b8d1e745df21f90","entry":"format","repo":"SteveTANTAN/PoG","repo_kind":"listed","path":"PoG/freebase_func.py","file_url":"https://github.com/SteveTANTAN/PoG/blob/HEAD/PoG/freebase_func.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4b8d1e745df21f90"}},{"code_sha256_prefix":"2506a6cd6f20658c","entry":"load_from_large_db","repo":"SteveTANTAN/PoG","repo_kind":"listed","path":"PoG/subgraph_utilts.py","file_url":"https://github.com/SteveTANTAN/PoG/blob/HEAD/PoG/subgraph_utilts.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2506a6cd6f20658c"}},{"code_sha256_prefix":"51dbd68f4d5aba9e","entry":"num_tokens_from_string","repo":"SteveTANTAN/PoG","repo_kind":"listed","path":"PoG/PoG_multi.py","file_url":"https://github.com/SteveTANTAN/PoG/blob/HEAD/PoG/PoG_multi.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"51dbd68f4d5aba9e"}},{"code_sha256_prefix":"68fca283c8ee65c1","entry":"prepare_dataset","repo":"SteveTANTAN/PoG","repo_kind":"listed","path":"PoG/utils.py","file_url":"https://github.com/SteveTANTAN/PoG/blob/HEAD/PoG/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"68fca283c8ee65c1"}},{"code_sha256_prefix":"32b462cabe595aae","entry":"retry_operation","repo":"SteveTANTAN/PoG","repo_kind":"listed","path":"PoG/subgraph_utilts.py","file_url":"https://github.com/SteveTANTAN/PoG/blob/HEAD/PoG/subgraph_utilts.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"32b462cabe595aae"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}