{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/fusegen-plm-fusion-for-data-generation-based","title":"FuseGen: PLM Fusion for Data-generation based Zero-shot Learning","arxiv_id":"2406.12527","date":"2024-06-18","proceeding":null,"authors":["Tianyuan Zou","Yang Liu","Peng Li","Jianqing Zhang","Jingjing Liu","Ya-Qin Zhang"],"abstract":"Data generation-based zero-shot learning, although effective in training Small Task-specific Models (STMs) via synthetic datasets generated by Pre-trained Language Models (PLMs), is often limited by the low quality of such synthetic datasets. Previous solutions have primarily focused on single PLM settings, where synthetic datasets are typically restricted to specific sub-spaces and often deviate from real-world distributions, leading to severe distribution bias. To mitigate such bias, we propose FuseGen, a novel data generation-based zero-shot learning framework that introduces a new criteria for subset selection from synthetic datasets via utilizing multiple PLMs and trained STMs. The chosen subset provides in-context feedback to each PLM, enhancing dataset quality through iterative data generation. Trained STMs are then used for sample re-weighting as well, further improving data quality. Extensive experiments across diverse tasks demonstrate that FuseGen substantially outperforms existing methods, highly effective in boosting STM performance in a PLM-agnostic way. Code is provided in https://github.com/LindaLydia/FuseGen.","url_abs":"https://arxiv.org/abs/2406.12527v1","url_pdf":"https://arxiv.org/pdf/2406.12527v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"fusegen-plm-fusion-for-data-generation-based","repo_url":"https://github.com/LindaLydia/FuseGen","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"zero-shot-learning","task_name":"Zero-Shot Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.12527","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.12527"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/yumeng5/SuperGen","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":6,"ran_honours":1,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":8,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"141522bb11524e19","entry":"combine","repo":"yumeng5/SuperGen","repo_kind":"found_in_text","path":"src/gen_utils.py","file_url":"https://github.com/yumeng5/SuperGen/blob/HEAD/src/gen_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"141522bb11524e19"}},{"code_sha256_prefix":"e448786d5ff1875a","entry":"default_dev_objective","repo":"yumeng5/SuperGen","repo_kind":"found_in_text","path":"src/trainer.py","file_url":"https://github.com/yumeng5/SuperGen/blob/HEAD/src/trainer.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e448786d5ff1875a"}},{"code_sha256_prefix":"e768e25ca95f70cd","entry":"input_example_to_string","repo":"yumeng5/SuperGen","repo_kind":"found_in_text","path":"src/dataset.py","file_url":"https://github.com/yumeng5/SuperGen/blob/HEAD/src/dataset.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e768e25ca95f70cd"}},{"code_sha256_prefix":"fbef70dc7c700501","entry":"input_example_to_tuple","repo":"yumeng5/SuperGen","repo_kind":"found_in_text","path":"src/dataset.py","file_url":"https://github.com/yumeng5/SuperGen/blob/HEAD/src/dataset.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fbef70dc7c700501"}},{"code_sha256_prefix":"8657d316b2a829b2","entry":"read_files","repo":"yumeng5/SuperGen","repo_kind":"found_in_text","path":"src/gen_utils.py","file_url":"https://github.com/yumeng5/SuperGen/blob/HEAD/src/gen_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8657d316b2a829b2"}},{"code_sha256_prefix":"56a2f49e4e3b9f21","entry":"relative_position_bucket","repo":"yumeng5/SuperGen","repo_kind":"found_in_text","path":"cocolm/modeling_cocolm.py","file_url":"https://github.com/yumeng5/SuperGen/blob/HEAD/cocolm/modeling_cocolm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"56a2f49e4e3b9f21"}},{"code_sha256_prefix":"9d3f9758d04d8a8d","entry":"sort_score","repo":"yumeng5/SuperGen","repo_kind":"found_in_text","path":"src/gen_utils.py","file_url":"https://github.com/yumeng5/SuperGen/blob/HEAD/src/gen_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9d3f9758d04d8a8d"}},{"code_sha256_prefix":"a23897d8e4196682","entry":"tokenize_multipart_input","repo":"yumeng5/SuperGen","repo_kind":"found_in_text","path":"src/dataset.py","file_url":"https://github.com/yumeng5/SuperGen/blob/HEAD/src/dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a23897d8e4196682"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}