{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/metasql-a-generate-then-rank-framework-for","title":"Metasql: A Generate-then-Rank Framework for Natural Language to SQL Translation","arxiv_id":"2402.17144","date":"2024-02-27","proceeding":null,"authors":["Yuankai Fan","Zhenying He","Tonghui Ren","Can Huang","Yinan Jing","Kai Zhang","X. Sean Wang"],"abstract":"The Natural Language Interface to Databases (NLIDB) empowers non-technical users with database access through intuitive natural language (NL) interactions. Advanced approaches, utilizing neural sequence-to-sequence models or large-scale language models, typically employ auto-regressive decoding to generate unique SQL queries sequentially. While these translation models have greatly improved the overall translation accuracy, surpassing 70% on NLIDB benchmarks, the use of auto-regressive decoding to generate single SQL queries may result in sub-optimal outputs, potentially leading to erroneous translations. In this paper, we propose Metasql, a unified generate-then-rank framework that can be flexibly incorporated with existing NLIDBs to consistently improve their translation accuracy. Metasql introduces query metadata to control the generation of better SQL query candidates and uses learning-to-rank algorithms to retrieve globally optimized queries. Specifically, Metasql first breaks down the meaning of the given NL query into a set of possible query metadata, representing the basic concepts of the semantics. These metadata are then used as language constraints to steer the underlying translation model toward generating a set of candidate SQL queries. Finally, Metasql ranks the candidates to identify the best matching one for the given NL query. Extensive experiments are performed to study Metasql on two public NLIDB benchmarks. The results show that the performance of the translation models can be effectively improved using Metasql.","url_abs":"https://arxiv.org/abs/2402.17144v1","url_pdf":"https://arxiv.org/pdf/2402.17144v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"metasql-a-generate-then-rank-framework-for","repo_url":"https://github.com/Kaimary/MetaSQL","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"learning-to-rank","task_name":"Learning-To-Rank"},{"task_slug":"translation","task_name":"Translation"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2402.17144","atlas_url":"https://app.syntology.ai/?focus=2402.17144","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.17144"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Kaimary/MetaSQL","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":7,"ran_draft_wrong":2,"unverified":3},"by_repo_kind":{"official":{"samples":12,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"00436a8a2dd58cad","entry":"candidates_sqlization","repo":"Kaimary/MetaSQL","repo_kind":"official","path":"allenmodels/dataset_readers/listwise_pair_reader_ratsql.py","file_url":"https://github.com/Kaimary/MetaSQL/blob/HEAD/allenmodels/dataset_readers/listwise_pair_reader_ratsql.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"00436a8a2dd58cad"}},{"code_sha256_prefix":"6bef2da47f1a8400","entry":"clamp","repo":"Kaimary/MetaSQL","repo_kind":"official","path":"allenmodels/dataset_readers/enc_preproc.py","file_url":"https://github.com/Kaimary/MetaSQL/blob/HEAD/allenmodels/dataset_readers/enc_preproc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6bef2da47f1a8400"}},{"code_sha256_prefix":"ac13caa2ecfa34fe","entry":"clones","repo":"Kaimary/MetaSQL","repo_kind":"official","path":"allenmodels/modules/transformer.py","file_url":"https://github.com/Kaimary/MetaSQL/blob/HEAD/allenmodels/modules/transformer.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ac13caa2ecfa34fe"}},{"code_sha256_prefix":"5ef8c55289697887","entry":"creating_schema","repo":"Kaimary/MetaSQL","repo_kind":"official","path":"openai_generate_with_meta.py","file_url":"https://github.com/Kaimary/MetaSQL/blob/HEAD/openai_generate_with_meta.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5ef8c55289697887"}},{"code_sha256_prefix":"d3104f05227a678b","entry":"load_schema","repo":"Kaimary/MetaSQL","repo_kind":"official","path":"allenmodels/dataset_readers/listwise_pair_reader_ratsql.py","file_url":"https://github.com/Kaimary/MetaSQL/blob/HEAD/allenmodels/dataset_readers/listwise_pair_reader_ratsql.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d3104f05227a678b"}},{"code_sha256_prefix":"b1447ba09556aed5","entry":"relative_attention_values","repo":"Kaimary/MetaSQL","repo_kind":"official","path":"allenmodels/modules/transformer.py","file_url":"https://github.com/Kaimary/MetaSQL/blob/HEAD/allenmodels/modules/transformer.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b1447ba09556aed5"}},{"code_sha256_prefix":"177a4d3341a6623f","entry":"table_text_encoding","repo":"Kaimary/MetaSQL","repo_kind":"official","path":"allenmodels/dataset_readers/multi_label_classifier_reader.py","file_url":"https://github.com/Kaimary/MetaSQL/blob/HEAD/allenmodels/dataset_readers/multi_label_classifier_reader.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"177a4d3341a6623f"}},{"code_sha256_prefix":"ba31bb11956f3909","entry":"to_dict_with_sorted_values","repo":"Kaimary/MetaSQL","repo_kind":"official","path":"allenmodels/dataset_readers/enc_preproc.py","file_url":"https://github.com/Kaimary/MetaSQL/blob/HEAD/allenmodels/dataset_readers/enc_preproc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ba31bb11956f3909"}},{"code_sha256_prefix":"3be1fa204837231a","entry":"tree_reduce","repo":"Kaimary/MetaSQL","repo_kind":"official","path":"allenmodels/dataset_readers/listwise_pair_reader_ratsql.py","file_url":"https://github.com/Kaimary/MetaSQL/blob/HEAD/allenmodels/dataset_readers/listwise_pair_reader_ratsql.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3be1fa204837231a"}},{"code_sha256_prefix":"aebf06ad16efc68b","entry":"find_foreign_keys_MYSQL_like","repo":"Kaimary/MetaSQL","repo_kind":"official","path":"openai_generate_with_meta.py","file_url":"https://github.com/Kaimary/MetaSQL/blob/HEAD/openai_generate_with_meta.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"aebf06ad16efc68b"}},{"code_sha256_prefix":"9693d8bfedca49c8","entry":"find_primary_keys_MYSQL_like","repo":"Kaimary/MetaSQL","repo_kind":"official","path":"openai_generate_with_meta.py","file_url":"https://github.com/Kaimary/MetaSQL/blob/HEAD/openai_generate_with_meta.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9693d8bfedca49c8"}},{"code_sha256_prefix":"ccc72a1f79c00ef3","entry":"relative_attention_logits","repo":"Kaimary/MetaSQL","repo_kind":"official","path":"allenmodels/modules/transformer.py","file_url":"https://github.com/Kaimary/MetaSQL/blob/HEAD/allenmodels/modules/transformer.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ccc72a1f79c00ef3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}