{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/hi-arg-exploring-the-integration-of","title":"Hi-ArG: Exploring the Integration of Hierarchical Argumentation Graphs in Language Pretraining","arxiv_id":"2312.00874","date":"2023-12-01","proceeding":null,"authors":["Jingcong Liang","Rong Ye","Meng Han","Qi Zhang","Ruofei Lai","Xinyu Zhang","Zhao Cao","Xuanjing Huang","Zhongyu Wei"],"abstract":"The knowledge graph is a structure to store and represent knowledge, and recent studies have discussed its capability to assist language models for various applications. Some variations of knowledge graphs aim to record arguments and their relations for computational argumentation tasks. However, many must simplify semantic types to fit specific schemas, thus losing flexibility and expression ability. In this paper, we propose the Hierarchical Argumentation Graph (Hi-ArG), a new structure to organize arguments. We also introduce two approaches to exploit Hi-ArG, including a text-graph multi-modal model GreaseArG and a new pre-training framework augmented with graph information. Experiments on two argumentation tasks have shown that after further pre-training and fine-tuning, GreaseArG supersedes same-scale language models on these tasks, while incorporating graph information during further pre-training can also improve the performance of vanilla language models. Code for this paper is available at https://github.com/ljcleo/Hi-ArG .","url_abs":"https://arxiv.org/abs/2312.00874v1","url_pdf":"https://arxiv.org/pdf/2312.00874v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"hi-arg-exploring-the-integration-of","repo_url":"https://github.com/ljcleo/hi-arg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"knowledge-graphs","task_name":"Knowledge Graphs"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2312.00874","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.00874"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/ljcleo/Hi-ArG","reach":null}],"summary":{"ran":2,"ran_draft_wrong":1,"unverified":6},"by_repo_kind":{"official":{"samples":9,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5d05826a61265e4c","entry":"GreaseArgGraphEmbeddings","repo":"ljcleo/Hi-ArG","repo_kind":"official","path":"argsme/pretrain/model.py","file_url":"https://github.com/ljcleo/Hi-ArG/blob/HEAD/argsme/pretrain/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5d05826a61265e4c"}},{"code_sha256_prefix":"5530c2f837ba74df","entry":"GreaseArgMixLayer","repo":"ljcleo/Hi-ArG","repo_kind":"official","path":"argsme/pretrain/model.py","file_url":"https://github.com/ljcleo/Hi-ArG/blob/HEAD/argsme/pretrain/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5530c2f837ba74df"}},{"code_sha256_prefix":"5c02dbcbfa96913e","entry":"get_activation","repo":"ljcleo/Hi-ArG","repo_kind":"official","path":"argsme/pretrain/model.py","file_url":"https://github.com/ljcleo/Hi-ArG/blob/HEAD/argsme/pretrain/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5c02dbcbfa96913e"}},{"code_sha256_prefix":"b641adcc36eab75c","entry":"GreaseArgConfig","repo":"ljcleo/Hi-ArG","repo_kind":"official","path":"argsme/pretrain/model.py","file_url":"https://github.com/ljcleo/Hi-ArG/blob/HEAD/argsme/pretrain/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b641adcc36eab75c"}},{"code_sha256_prefix":"9a71a406f45cab83","entry":"GreaseArgGraphTransformerLayer","repo":"ljcleo/Hi-ArG","repo_kind":"official","path":"argsme/pretrain/model.py","file_url":"https://github.com/ljcleo/Hi-ArG/blob/HEAD/argsme/pretrain/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a71a406f45cab83"}},{"code_sha256_prefix":"b27c3f11b5b30230","entry":"GreaseArgModel","repo":"ljcleo/Hi-ArG","repo_kind":"official","path":"argsme/pretrain/model.py","file_url":"https://github.com/ljcleo/Hi-ArG/blob/HEAD/argsme/pretrain/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b27c3f11b5b30230"}},{"code_sha256_prefix":"8afbb153ed089875","entry":"GreaseArgMutualAttention","repo":"ljcleo/Hi-ArG","repo_kind":"official","path":"argsme/pretrain/model.py","file_url":"https://github.com/ljcleo/Hi-ArG/blob/HEAD/argsme/pretrain/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8afbb153ed089875"}},{"code_sha256_prefix":"d9cda6476cc664ec","entry":"GreaseArgPreTrainedModel","repo":"ljcleo/Hi-ArG","repo_kind":"official","path":"argsme/pretrain/model.py","file_url":"https://github.com/ljcleo/Hi-ArG/blob/HEAD/argsme/pretrain/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d9cda6476cc664ec"}},{"code_sha256_prefix":"7429bcc3694be456","entry":"GreaseArgTextGraphAttention","repo":"ljcleo/Hi-ArG","repo_kind":"official","path":"argsme/pretrain/model.py","file_url":"https://github.com/ljcleo/Hi-ArG/blob/HEAD/argsme/pretrain/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7429bcc3694be456"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}