{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/llm4dyg-can-large-language-models-solve","title":"LLM4DyG: Can Large Language Models Solve Spatial-Temporal Problems on Dynamic Graphs?","arxiv_id":"2310.17110","date":"2023-10-26","proceeding":null,"authors":["Zeyang Zhang","Xin Wang","Ziwei Zhang","Haoyang Li","Yijian Qin","Wenwu Zhu"],"abstract":"In an era marked by the increasing adoption of Large Language Models (LLMs) for various tasks, there is a growing focus on exploring LLMs' capabilities in handling web data, particularly graph data. Dynamic graphs, which capture temporal network evolution patterns, are ubiquitous in real-world web data. Evaluating LLMs' competence in understanding spatial-temporal information on dynamic graphs is essential for their adoption in web applications, which remains unexplored in the literature. In this paper, we bridge the gap via proposing to evaluate LLMs' spatial-temporal understanding abilities on dynamic graphs, to the best of our knowledge, for the first time. Specifically, we propose the LLM4DyG benchmark, which includes nine specially designed tasks considering the capability evaluation of LLMs from both temporal and spatial dimensions. Then, we conduct extensive experiments to analyze the impacts of different data generators, data statistics, prompting techniques, and LLMs on the model performance. Finally, we propose Disentangled Spatial-Temporal Thoughts (DST2) for LLMs on dynamic graphs to enhance LLMs' spatial-temporal understanding abilities. Our main observations are: 1) LLMs have preliminary spatial-temporal understanding abilities on dynamic graphs, 2) Dynamic graph tasks show increasing difficulties for LLMs as the graph size and density increase, while not sensitive to the time span and data generation mechanism, 3) the proposed DST2 prompting method can help to improve LLMs' spatial-temporal understanding abilities on dynamic graphs for most tasks. The data and codes are publicly available at Github.","url_abs":"https://arxiv.org/abs/2310.17110v3","url_pdf":"https://arxiv.org/pdf/2310.17110v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"llm4dyg-can-large-language-models-solve","repo_url":"https://github.com/wondergo2017/llm4dyg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2310.17110","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2310.17110"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wondergo2017/llm4dyg","reach":{"status":"ok"}}],"summary":{"ran":12,"unverified":3},"by_repo_kind":{"official":{"samples":15,"ran":12,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":15,"samples":[{"code_sha256_prefix":"eaafd6210492f18a","entry":"cal_metric","repo":"wondergo2017/llm4dyg","repo_kind":"official","path":"llm4dyg/utils/task/link_pred.py","file_url":"https://github.com/wondergo2017/llm4dyg/blob/HEAD/llm4dyg/utils/task/link_pred.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eaafd6210492f18a"}},{"code_sha256_prefix":"472cf37c844678cc","entry":"find_edge_t","repo":"wondergo2017/llm4dyg","repo_kind":"official","path":"llm4dyg/utils/task/check_tclosure.py","file_url":"https://github.com/wondergo2017/llm4dyg/blob/HEAD/llm4dyg/utils/task/check_tclosure.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"472cf37c844678cc"}},{"code_sha256_prefix":"0a10961fd691c1fa","entry":"find_triangles","repo":"wondergo2017/llm4dyg","repo_kind":"official","path":"llm4dyg/utils/task/check_tclosure.py","file_url":"https://github.com/wondergo2017/llm4dyg/blob/HEAD/llm4dyg/utils/task/check_tclosure.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0a10961fd691c1fa"}},{"code_sha256_prefix":"17303501ce8dd466","entry":"format","repo":"wondergo2017/llm4dyg","repo_kind":"official","path":"llm4dyg/utils/misc.py","file_url":"https://github.com/wondergo2017/llm4dyg/blob/HEAD/llm4dyg/utils/misc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"17303501ce8dd466"}},{"code_sha256_prefix":"38b1970e3236cd09","entry":"generate_start_node","repo":"wondergo2017/llm4dyg","repo_kind":"official","path":"llm4dyg/utils/task/find_tpath.py","file_url":"https://github.com/wondergo2017/llm4dyg/blob/HEAD/llm4dyg/utils/task/find_tpath.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"38b1970e3236cd09"}},{"code_sha256_prefix":"a421a22efb08625e","entry":"get_imp","repo":"wondergo2017/llm4dyg","repo_kind":"official","path":"llm4dyg/utils/prompt.py","file_url":"https://github.com/wondergo2017/llm4dyg/blob/HEAD/llm4dyg/utils/prompt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a421a22efb08625e"}},{"code_sha256_prefix":"a224653905322982","entry":"get_sbm_graph","repo":"wondergo2017/llm4dyg","repo_kind":"official","path":"llm4dyg/utils/data.py","file_url":"https://github.com/wondergo2017/llm4dyg/blob/HEAD/llm4dyg/utils/data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a224653905322982"}},{"code_sha256_prefix":"e358a88db03848af","entry":"judge_path","repo":"wondergo2017/llm4dyg","repo_kind":"official","path":"llm4dyg/utils/task/check_tclosure.py","file_url":"https://github.com/wondergo2017/llm4dyg/blob/HEAD/llm4dyg/utils/task/check_tclosure.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e358a88db03848af"}},{"code_sha256_prefix":"d4a5f3a62c2d5e51","entry":"judge_path","repo":"wondergo2017/llm4dyg","repo_kind":"official","path":"llm4dyg/utils/task/check_tpath.py","file_url":"https://github.com/wondergo2017/llm4dyg/blob/HEAD/llm4dyg/utils/task/check_tpath.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d4a5f3a62c2d5e51"}},{"code_sha256_prefix":"f4cbd1198b579dc1","entry":"select","repo":"wondergo2017/llm4dyg","repo_kind":"official","path":"llm4dyg/utils/task/check_tpath.py","file_url":"https://github.com/wondergo2017/llm4dyg/blob/HEAD/llm4dyg/utils/task/check_tpath.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f4cbd1198b579dc1"}},{"code_sha256_prefix":"2b6bc118c9a80df5","entry":"turn3dm","repo":"wondergo2017/llm4dyg","repo_kind":"official","path":"llm4dyg/utils/data.py","file_url":"https://github.com/wondergo2017/llm4dyg/blob/HEAD/llm4dyg/utils/data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2b6bc118c9a80df5"}},{"code_sha256_prefix":"5c16f00fad3e59aa","entry":"turn_undirect","repo":"wondergo2017/llm4dyg","repo_kind":"official","path":"llm4dyg/utils/data.py","file_url":"https://github.com/wondergo2017/llm4dyg/blob/HEAD/llm4dyg/utils/data.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5c16f00fad3e59aa"}},{"code_sha256_prefix":"fd033bebc1ed43d0","entry":"send_to_chat","repo":"wondergo2017/llm4dyg","repo_kind":"official","path":"llm4dyg/utils/api.py","file_url":"https://github.com/wondergo2017/llm4dyg/blob/HEAD/llm4dyg/utils/api.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fd033bebc1ed43d0"}},{"code_sha256_prefix":"8d8c8f4f5c9ba1e9","entry":"send_to_chat_gpt","repo":"wondergo2017/llm4dyg","repo_kind":"official","path":"llm4dyg/utils/api.py","file_url":"https://github.com/wondergo2017/llm4dyg/blob/HEAD/llm4dyg/utils/api.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8d8c8f4f5c9ba1e9"}},{"code_sha256_prefix":"19fccff2aaf4c9bd","entry":"send_to_chat_gpt_intruct","repo":"wondergo2017/llm4dyg","repo_kind":"official","path":"llm4dyg/utils/api.py","file_url":"https://github.com/wondergo2017/llm4dyg/blob/HEAD/llm4dyg/utils/api.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"19fccff2aaf4c9bd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}