{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/omnitab-pretraining-with-natural-and-1","title":"OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering","arxiv_id":"2207.03637","date":"2022-07-08","proceeding":"NAACL 2022 7","authors":["Zhengbao Jiang","Yi Mao","Pengcheng He","Graham Neubig","Weizhu Chen"],"abstract":"The information in tables can be an important complement to text, making table-based question answering (QA) systems of great value. The intrinsic complexity of handling tables often adds an extra burden to both model design and data annotation. In this paper, we aim to develop a simple table-based QA model with minimal annotation effort. Motivated by the fact that table-based QA requires both alignment between questions and tables and the ability to perform complicated reasoning over multiple table elements, we propose an omnivorous pretraining approach that consumes both natural and synthetic data to endow models with these respective abilities. Specifically, given freely available tables, we leverage retrieval to pair them with relevant natural sentences for mask-based pretraining, and synthesize NL questions by converting SQL sampled from tables for pretraining with a QA loss. We perform extensive experiments in both few-shot and full settings, and the results clearly demonstrate the superiority of our model OmniTab, with the best multitasking approach achieving an absolute gain of 16.2% and 2.7% in 128-shot and full settings respectively, also establishing a new state-of-the-art on WikiTableQuestions. Detailed ablations and analyses reveal different characteristics of natural and synthetic data, shedding light on future directions in omnivorous pretraining. Code, pretraining data, and pretrained models are available at https://github.com/jzbjyb/OmniTab.","url_abs":"https://arxiv.org/abs/2207.03637v1","url_pdf":"https://arxiv.org/pdf/2207.03637v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"omnitab-pretraining-with-natural-and-1","repo_url":"https://github.com/jzbjyb/omnitab","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"semantic-parsing","task_name":"Semantic Parsing"},{"task_slug":"table-based-question-answering","task_name":"Table-based Question Answering"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/semantic-parsing-on-wikitablequestions","task":"Semantic Parsing","dataset":"WikiTableQuestions","model":"OmniTab-Large","rank_in_archive_order":14,"of":22,"metrics":{"Accuracy (Dev)":"62.5","Accuracy (Test)":"63.3"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2207.03637","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2207.03637"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jzbjyb/omnitab","reach":null}],"summary":{"ran":1,"unverified":1},"by_repo_kind":{"official":{"samples":2,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"03dce1d1f30541d5","entry":"IndexedRowTableLinearizeOffset","repo":"jzbjyb/omnitab","repo_kind":"official","path":"tokenization_omnitab.py","file_url":"https://github.com/jzbjyb/omnitab/blob/HEAD/tokenization_omnitab.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"03dce1d1f30541d5"}},{"code_sha256_prefix":"d854344dcf4804e1","entry":"OmnitabTokenizer","repo":"jzbjyb/omnitab","repo_kind":"official","path":"tokenization_omnitab.py","file_url":"https://github.com/jzbjyb/omnitab/blob/HEAD/tokenization_omnitab.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d854344dcf4804e1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}