{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unifiedskg-unifying-and-multi-tasking","title":"UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models","arxiv_id":"2201.05966","date":"2022-01-16","proceeding":null,"authors":["Tianbao Xie","Chen Henry Wu","Peng Shi","Ruiqi Zhong","Torsten Scholak","Michihiro Yasunaga","Chien-Sheng Wu","Ming Zhong","Pengcheng Yin","Sida I. Wang","Victor Zhong","Bailin Wang","Chengzu Li","Connor Boyle","Ansong Ni","Ziyu Yao","Dragomir Radev","Caiming Xiong","Lingpeng Kong","Rui Zhang","Noah A. Smith","Luke Zettlemoyer","Tao Yu"],"abstract":"Structured knowledge grounding (SKG) leverages structured knowledge to complete user requests, such as semantic parsing over databases and question answering over knowledge bases. Since the inputs and outputs of SKG tasks are heterogeneous, they have been studied separately by different communities, which limits systematic and compatible research on SKG. In this paper, we overcome this limitation by proposing the UnifiedSKG framework, which unifies 21 SKG tasks into a text-to-text format, aiming to promote systematic SKG research, instead of being exclusive to a single task, domain, or dataset. We use UnifiedSKG to benchmark T5 with different sizes and show that T5, with simple modifications when necessary, achieves state-of-the-art performance on almost all of the 21 tasks. We further demonstrate that multi-task prefix-tuning improves the performance on most tasks, largely improving the overall performance. UnifiedSKG also facilitates the investigation of zero-shot and few-shot learning, and we show that T0, GPT-3, and Codex struggle in zero-shot and few-shot learning for SKG. We also use UnifiedSKG to conduct a series of controlled experiments on structured knowledge encoding variants across SKG tasks. UnifiedSKG is easily extensible to more tasks, and it is open-sourced at https://github.com/hkunlp/unifiedskg.","url_abs":"https://arxiv.org/abs/2201.05966v3","url_pdf":"https://arxiv.org/pdf/2201.05966v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unifiedskg-unifying-and-multi-tasking","repo_url":"https://github.com/hkunlp/unifiedskg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"semantic-parsing","task_name":"Semantic Parsing"},{"task_slug":"table-based-fact-verification","task_name":"Table-based Fact Verification"},{"task_slug":"task-oriented-dialogue-systems","task_name":"Task-Oriented Dialogue Systems"}],"methods":[{"method_slug":"adafactor","method_name":"Adafactor"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"cosine-annealing","method_name":"Cosine Annealing"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"gpt-3","method_name":"GPT-3"},{"method_slug":"glu","method_name":"Gated Linear Unit"},{"method_slug":"inverse-square-root-schedule","method_name":"Inverse Square Root Schedule"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-cosine-annealing","method_name":"Linear Warmup With Cosine Annealing"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"sentencepiece","method_name":"SentencePiece"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"t5","method_name":"T5"},{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/semantic-parsing-on-wikitablequestions","task":"Semantic Parsing","dataset":"WikiTableQuestions","model":"T5-3b(UnifiedSKG)","rank_in_archive_order":19,"of":22,"metrics":{"Accuracy (Dev)":"50.65","Accuracy (Test)":"49.29"},"uses_additional_data":false},{"leaderboard":"/sota/table-based-fact-verification-on-tabfact","task":"Table-based Fact Verification","dataset":"TabFact","model":"T5-3b(UnifiedSKG)","rank_in_archive_order":9,"of":15,"metrics":{"Test":"83.68","Val":"83.97"},"uses_additional_data":false},{"leaderboard":"/sota/task-oriented-dialogue-systems-on-kvret","task":"Task-Oriented Dialogue Systems","dataset":"KVRET","model":"T5-3b(UnifiedSKG)","rank_in_archive_order":1,"of":10,"metrics":{"Entity F1":"70.07"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2201.05966","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2201.05966"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/hkunlp/unifiedskg","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":1,"unverified":4},"by_repo_kind":{"official":{"samples":5,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"a315b8fc32a18efe","entry":"shift_tokens_right","repo":"hkunlp/unifiedskg","repo_kind":"official","path":"models/prompt/modeling_bart.py","file_url":"https://github.com/hkunlp/unifiedskg/blob/HEAD/models/prompt/modeling_bart.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a315b8fc32a18efe"}},{"code_sha256_prefix":"ff9289c16a83589c","entry":"aggregate_prompt","repo":"hkunlp/unifiedskg","repo_kind":"official","path":"models/unified/combined_prefixtuning.py","file_url":"https://github.com/hkunlp/unifiedskg/blob/HEAD/models/unified/combined_prefixtuning.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ff9289c16a83589c"}},{"code_sha256_prefix":"eb8d0541d7792310","entry":"compute_interaction_metric","repo":"hkunlp/unifiedskg","repo_kind":"official","path":"metrics/sparc/interaction_scores.py","file_url":"https://github.com/hkunlp/unifiedskg/blob/HEAD/metrics/sparc/interaction_scores.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"eb8d0541d7792310"}},{"code_sha256_prefix":"e8a3b4f10de70271","entry":"evaluate","repo":"hkunlp/unifiedskg","repo_kind":"official","path":"metrics/sparc/interaction_scores.py","file_url":"https://github.com/hkunlp/unifiedskg/blob/HEAD/metrics/sparc/interaction_scores.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e8a3b4f10de70271"}},{"code_sha256_prefix":"136abfa3e8ce028d","entry":"load_tf_weights_in_t5","repo":"hkunlp/unifiedskg","repo_kind":"official","path":"models/adapter/modeling_t5.py","file_url":"https://github.com/hkunlp/unifiedskg/blob/HEAD/models/adapter/modeling_t5.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"136abfa3e8ce028d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}