{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/rendering-graphs-for-graph-reasoning-in","title":"GITA: Graph to Visual and Textual Integration for Vision-Language Graph Reasoning","arxiv_id":"2402.02130","date":"2024-02-03","proceeding":null,"authors":["Yanbin Wei","Shuai Fu","Weisen Jiang","Zejian Zhang","Zhixiong Zeng","Qi Wu","James T. Kwok","Yu Zhang"],"abstract":"Large Language Models (LLMs) are increasingly used for various tasks with graph structures. Though LLMs can process graph information in a textual format, they overlook the rich vision modality, which is an intuitive way for humans to comprehend structural information and conduct general graph reasoning. The potential benefits and capabilities of representing graph structures as visual images (i.e., $\\textit{visual graph}$) are still unexplored. To fill the gap, we innovatively propose an end-to-end framework, called $\\textbf{G}$raph to v$\\textbf{I}$sual and $\\textbf{T}$extual Integr$\\textbf{A}$tion (GITA), which firstly incorporates visual graphs into general graph reasoning. Besides, we establish $\\textbf{G}$raph-based $\\textbf{V}$ision-$\\textbf{L}$anguage $\\textbf{Q}$uestion $\\textbf{A}$nswering (GVLQA) dataset from existing graph data, which is the first vision-language dataset for general graph reasoning purposes. Extensive experiments on the GVLQA dataset and five real-world datasets show that GITA outperforms mainstream LLMs in terms of general graph reasoning capabilities. Moreover, We highlight the effectiveness of the layout augmentation on visual graphs and pretraining on the GVLQA dataset.","url_abs":"https://arxiv.org/abs/2402.02130v5","url_pdf":"https://arxiv.org/pdf/2402.02130v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"rendering-graphs-for-graph-reasoning-in","repo_url":"https://github.com/WEIYanbin1999/GITA","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"link-prediction","task_name":"Link Prediction"},{"task_slug":"node-classification","task_name":"Node Classification"}],"methods":[],"datasets_introduced":[{"slug":"gvlqa","name":"GVLQA","full_name":"Graph Vision-Language Question-Answering"}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.02130","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.02130"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/WEIYanbin1999/GITA","reach":{"status":"ok"}}],"summary":{"ran":2,"ran_violates":1,"unverified":7},"by_repo_kind":{"official":{"samples":10,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"fa6e5d1f1da79178","entry":"add_mask","repo":"WEIYanbin1999/GITA","repo_kind":"official","path":"fastchat/linkpred_builder.py","file_url":"https://github.com/WEIYanbin1999/GITA/blob/HEAD/fastchat/linkpred_builder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fa6e5d1f1da79178"}},{"code_sha256_prefix":"7f57cd72ced7728d","entry":"create_data_object","repo":"WEIYanbin1999/GITA","repo_kind":"official","path":"fastchat/linkpred_builder.py","file_url":"https://github.com/WEIYanbin1999/GITA/blob/HEAD/fastchat/linkpred_builder.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7f57cd72ced7728d"}},{"code_sha256_prefix":"0bf414a2b44babe6","entry":"raise_warning_for_incompatible_cpu_offloading_configuration","repo":"WEIYanbin1999/GITA","repo_kind":"official","path":"fastchat/model/model_adapter.py","file_url":"https://github.com/WEIYanbin1999/GITA/blob/HEAD/fastchat/model/model_adapter.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0bf414a2b44babe6"}},{"code_sha256_prefix":"80dde39da1ba3708","entry":"compress","repo":"WEIYanbin1999/GITA","repo_kind":"official","path":"fastchat/model/compression.py","file_url":"https://github.com/WEIYanbin1999/GITA/blob/HEAD/fastchat/model/compression.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"80dde39da1ba3708"}},{"code_sha256_prefix":"c5824da57107fe8c","entry":"get_compressed_list","repo":"WEIYanbin1999/GITA","repo_kind":"official","path":"fastchat/model/compression.py","file_url":"https://github.com/WEIYanbin1999/GITA/blob/HEAD/fastchat/model/compression.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c5824da57107fe8c"}},{"code_sha256_prefix":"25f1c2de6ce6ae79","entry":"get_conv_template","repo":"WEIYanbin1999/GITA","repo_kind":"official","path":"fastchat/conversation.py","file_url":"https://github.com/WEIYanbin1999/GITA/blob/HEAD/fastchat/conversation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"25f1c2de6ce6ae79"}},{"code_sha256_prefix":"2602e95732dab27b","entry":"link_prediction_custom_split","repo":"WEIYanbin1999/GITA","repo_kind":"official","path":"fastchat/linkpred_builder.py","file_url":"https://github.com/WEIYanbin1999/GITA/blob/HEAD/fastchat/linkpred_builder.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2602e95732dab27b"}},{"code_sha256_prefix":"5196732e9a84b6dc","entry":"load_compress_model","repo":"WEIYanbin1999/GITA","repo_kind":"official","path":"fastchat/model/compression.py","file_url":"https://github.com/WEIYanbin1999/GITA/blob/HEAD/fastchat/model/compression.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5196732e9a84b6dc"}},{"code_sha256_prefix":"dc21db7cf25bfe72","entry":"process_response","repo":"WEIYanbin1999/GITA","repo_kind":"official","path":"fastchat/model/model_chatglm.py","file_url":"https://github.com/WEIYanbin1999/GITA/blob/HEAD/fastchat/model/model_chatglm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dc21db7cf25bfe72"}},{"code_sha256_prefix":"e4f8545686f50507","entry":"recover_message_list","repo":"WEIYanbin1999/GITA","repo_kind":"official","path":"fastchat/model/model_chatglm.py","file_url":"https://github.com/WEIYanbin1999/GITA/blob/HEAD/fastchat/model/model_chatglm.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e4f8545686f50507"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}