{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/generating-data-for-symbolic-language-with","title":"Generating Data for Symbolic Language with Large Language Models","arxiv_id":"2305.13917","date":"2023-05-23","proceeding":null,"authors":["Jiacheng Ye","Chengzu Li","Lingpeng Kong","Tao Yu"],"abstract":"While large language models (LLMs) bring not only performance but also complexity, recent work has started to turn LLMs into data generators rather than task inferencers, where another affordable task model is trained for efficient deployment and inference. However, such an approach has primarily been applied to natural language tasks and has not yet been explored for symbolic language tasks with complex structured outputs (e.g., semantic parsing and code generation). In this paper, we propose SymGen which utilizes LLMs for generating various annotation-expensive symbolic language data. SymGen consists of an informative prompt to steer generation and an agreement-based verifier to improve data correctness. We conduct extensive experiments on six symbolic language tasks across various settings. Compared with the LLMs, we demonstrate the 1\\%-sized task model can achieve comparable or better performance, largely cutting inference and deployment costs. We also show that generated data with only a few human demonstrations can be as effective as over 10 times the amount of human-annotated data when training the task model, saving a considerable amount of annotation effort. SymGen sheds new light on data generation for complex tasks, and we release the code at \\href{https://github.com/HKUNLP/SymGen}{https://github.com/HKUNLP/SymGen}.","url_abs":"https://arxiv.org/abs/2305.13917v1","url_pdf":"https://arxiv.org/pdf/2305.13917v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"generating-data-for-symbolic-language-with","repo_url":"https://github.com/hkunlp/symgen","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"code-generation","task_name":"Code Generation"},{"task_slug":"semantic-parsing","task_name":"Semantic Parsing"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.13917","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.13917"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hkunlp/symgen","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/HKUNLP/SymGen","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":10},"by_repo_kind":{"official":{"samples":10,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"95e0731eb9afd765","entry":"by_a_post_process","repo":"hkunlp/symgen","repo_kind":"official","path":"src/filters/geoquery_filter.py","file_url":"https://github.com/hkunlp/symgen/blob/HEAD/src/filters/geoquery_filter.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"95e0731eb9afd765"}},{"code_sha256_prefix":"7cee44a5faf3bf92","entry":"by_dedup","repo":"hkunlp/symgen","repo_kind":"official","path":"src/filters/filter_tool.py","file_url":"https://github.com/hkunlp/symgen/blob/HEAD/src/filters/filter_tool.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7cee44a5faf3bf92"}},{"code_sha256_prefix":"91c673455ce64f8a","entry":"by_exec","repo":"HKUNLP/SymGen","repo_kind":"official","path":"src/filters/filter_tool.py","file_url":"https://github.com/HKUNLP/SymGen/blob/HEAD/src/filters/filter_tool.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"91c673455ce64f8a"}},{"code_sha256_prefix":"bd8a61f9fef03503","entry":"by_length","repo":"hkunlp/symgen","repo_kind":"official","path":"src/filters/filter_tool.py","file_url":"https://github.com/hkunlp/symgen/blob/HEAD/src/filters/filter_tool.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bd8a61f9fef03503"}},{"code_sha256_prefix":"46ac634191da2933","entry":"deduplicate","repo":"hkunlp/symgen","repo_kind":"official","path":"src/dataset_readers/index_dsr.py","file_url":"https://github.com/hkunlp/symgen/blob/HEAD/src/dataset_readers/index_dsr.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"46ac634191da2933"}},{"code_sha256_prefix":"f397316309d5028b","entry":"encode_field","repo":"hkunlp/symgen","repo_kind":"official","path":"src/dataset_readers/base_dsr.py","file_url":"https://github.com/hkunlp/symgen/blob/HEAD/src/dataset_readers/base_dsr.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f397316309d5028b"}},{"code_sha256_prefix":"d3e6bb8dcb712dfd","entry":"reassign_idx","repo":"hkunlp/symgen","repo_kind":"official","path":"src/dataset_readers/index_dsr.py","file_url":"https://github.com/hkunlp/symgen/blob/HEAD/src/dataset_readers/index_dsr.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d3e6bb8dcb712dfd"}},{"code_sha256_prefix":"e67773e4e43cf4c1","entry":"result_eq","repo":"hkunlp/symgen","repo_kind":"official","path":"src/filters/break_filter.py","file_url":"https://github.com/hkunlp/symgen/blob/HEAD/src/filters/break_filter.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e67773e4e43cf4c1"}},{"code_sha256_prefix":"01f68295484d4194","entry":"result_eq","repo":"hkunlp/symgen","repo_kind":"official","path":"src/filters/geoquery_filter.py","file_url":"https://github.com/hkunlp/symgen/blob/HEAD/src/filters/geoquery_filter.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"01f68295484d4194"}},{"code_sha256_prefix":"57cc9ebc64606720","entry":"text_to_field","repo":"hkunlp/symgen","repo_kind":"official","path":"src/filters/filter_tool.py","file_url":"https://github.com/hkunlp/symgen/blob/HEAD/src/filters/filter_tool.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"57cc9ebc64606720"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}