{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/text2sql-is-not-enough-unifying-ai-and","title":"Text2SQL is Not Enough: Unifying AI and Databases with TAG","arxiv_id":"2408.14717","date":"2024-08-27","proceeding":null,"authors":["Asim Biswal","Liana Patel","Siddarth Jha","Amog Kamsetty","Shu Liu","Joseph E. Gonzalez","Carlos Guestrin","Matei Zaharia"],"abstract":"AI systems that serve natural language questions over databases promise to unlock tremendous value. Such systems would allow users to leverage the powerful reasoning and knowledge capabilities of language models (LMs) alongside the scalable computational power of data management systems. These combined capabilities would empower users to ask arbitrary natural language questions over custom data sources. However, existing methods and benchmarks insufficiently explore this setting. Text2SQL methods focus solely on natural language questions that can be expressed in relational algebra, representing a small subset of the questions real users wish to ask. Likewise, Retrieval-Augmented Generation (RAG) considers the limited subset of queries that can be answered with point lookups to one or a few data records within the database. We propose Table-Augmented Generation (TAG), a unified and general-purpose paradigm for answering natural language questions over databases. The TAG model represents a wide range of interactions between the LM and database that have been previously unexplored and creates exciting research opportunities for leveraging the world knowledge and reasoning capabilities of LMs over data. We systematically develop benchmarks to study the TAG problem and find that standard methods answer no more than 20% of queries correctly, confirming the need for further research in this area. We release code for the benchmark at https://github.com/TAG-Research/TAG-Bench.","url_abs":"https://arxiv.org/abs/2408.14717v1","url_pdf":"https://arxiv.org/pdf/2408.14717v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"text2sql-is-not-enough-unifying-ai-and","repo_url":"https://github.com/tag-research/tag-bench","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"rag","task_name":"RAG"},{"task_slug":"retrieval-augmented-generation","task_name":"Retrieval-augmented Generation"},{"task_slug":"text-to-sql","task_name":"Text-To-SQL"},{"task_slug":"world-knowledge","task_name":"World Knowledge"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2408.14717","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2408.14717"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tag-research/tag-bench","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"3f3e9d727bcef0db","entry":"generate_comment_prompt","repo":"tag-research/tag-bench","repo_kind":"official","path":"setup/get_text2sql_prompts.py","file_url":"https://github.com/tag-research/tag-bench/blob/HEAD/setup/get_text2sql_prompts.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3f3e9d727bcef0db"}},{"code_sha256_prefix":"dc29c12b164751c7","entry":"generate_schema_prompt","repo":"tag-research/tag-bench","repo_kind":"official","path":"setup/get_text2sql_prompts.py","file_url":"https://github.com/tag-research/tag-bench/blob/HEAD/setup/get_text2sql_prompts.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dc29c12b164751c7"}},{"code_sha256_prefix":"08dfd0c28c8d8777","entry":"nice_look_table","repo":"tag-research/tag-bench","repo_kind":"official","path":"setup/get_text2sql_prompts.py","file_url":"https://github.com/tag-research/tag-bench/blob/HEAD/setup/get_text2sql_prompts.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"08dfd0c28c8d8777"}},{"code_sha256_prefix":"8f23a8874b3b0062","entry":"row_to_str","repo":"tag-research/tag-bench","repo_kind":"official","path":"tag/utils.py","file_url":"https://github.com/tag-research/tag-bench/blob/HEAD/tag/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8f23a8874b3b0062"}},{"code_sha256_prefix":"499ee2fe9d0bfcb6","entry":"run_row","repo":"tag-research/tag-bench","repo_kind":"official","path":"tag/text2sql.py","file_url":"https://github.com/tag-research/tag-bench/blob/HEAD/tag/text2sql.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"499ee2fe9d0bfcb6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}