{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/large-language-model-as-attributed-training-1","title":"Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias","arxiv_id":"2306.15895","date":"2023-06-28","proceeding":"NeurIPS 2023 11","authors":["Yue Yu","Yuchen Zhuang","Jieyu Zhang","Yu Meng","Alexander Ratner","Ranjay Krishna","Jiaming Shen","Chao Zhang"],"abstract":"Large language models (LLMs) have been recently leveraged as training data generators for various natural language processing (NLP) tasks. While previous research has explored different approaches to training models using generated data, they generally rely on simple class-conditional prompts, which may limit the diversity of the generated data and inherit systematic biases of LLM. Thus, we investigate training data generation with diversely attributed prompts (e.g., specifying attributes like length and style), which have the potential to yield diverse and attributed generated data. Our investigation focuses on datasets with high cardinality and diverse domains, wherein we demonstrate that attributed prompts outperform simple class-conditional prompts in terms of the resulting model's performance. Additionally, we present a comprehensive empirical study on data generation encompassing vital aspects like bias, diversity, and efficiency, and highlight three key observations: firstly, synthetic datasets generated by simple prompts exhibit significant biases, such as regional bias; secondly, attribute diversity plays a pivotal role in enhancing model performance; lastly, attributed prompts achieve the performance of simple class-conditional prompts while utilizing only 5\\% of the querying cost of ChatGPT associated with the latter. The data and code are available on \\url{https://github.com/yueyu1030/AttrPrompt}.","url_abs":"https://arxiv.org/abs/2306.15895v2","url_pdf":"https://arxiv.org/pdf/2306.15895v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"large-language-model-as-attributed-training-1","repo_url":"https://github.com/yueyu1030/attrprompt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"attribute","task_name":"Attribute"},{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"large-language-model","task_name":"Large Language Model"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2306.15895","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.15895"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/yueyu1030/AttrPrompt","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yueyu1030/attrprompt","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":4,"ran_draft_wrong":4,"unverified":9},"by_repo_kind":{"official":{"samples":17,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0aa551fedb8479cb","entry":"acc_and_f1","repo":"yueyu1030/attrprompt","repo_kind":"official","path":"train_classifier/plm_model/trainer.py","file_url":"https://github.com/yueyu1030/attrprompt/blob/HEAD/train_classifier/plm_model/trainer.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0aa551fedb8479cb"}},{"code_sha256_prefix":"2fd46a9fc43fa059","entry":"clean_str","repo":"yueyu1030/AttrPrompt","repo_kind":"official","path":"gen_train_data/filter.py","file_url":"https://github.com/yueyu1030/AttrPrompt/blob/HEAD/gen_train_data/filter.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2fd46a9fc43fa059"}},{"code_sha256_prefix":"efe1a86fd9a2468e","entry":"clean_str","repo":"yueyu1030/attrprompt","repo_kind":"official","path":"train_classifier/plm_model/utils.py","file_url":"https://github.com/yueyu1030/attrprompt/blob/HEAD/train_classifier/plm_model/utils.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"efe1a86fd9a2468e"}},{"code_sha256_prefix":"d851686037df7aca","entry":"compute_metrics","repo":"yueyu1030/attrprompt","repo_kind":"official","path":"train_classifier/plm_model/trainer.py","file_url":"https://github.com/yueyu1030/attrprompt/blob/HEAD/train_classifier/plm_model/trainer.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d851686037df7aca"}},{"code_sha256_prefix":"22baf81366f31381","entry":"compute_metrics_rel","repo":"yueyu1030/attrprompt","repo_kind":"official","path":"train_classifier/plm_model/trainer.py","file_url":"https://github.com/yueyu1030/attrprompt/blob/HEAD/train_classifier/plm_model/trainer.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"22baf81366f31381"}},{"code_sha256_prefix":"dd27c40f658addad","entry":"load_tokenizer","repo":"yueyu1030/attrprompt","repo_kind":"official","path":"train_classifier/plm_model/utils.py","file_url":"https://github.com/yueyu1030/attrprompt/blob/HEAD/train_classifier/plm_model/utils.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dd27c40f658addad"}},{"code_sha256_prefix":"77f69af078e09830","entry":"mean_pooling","repo":"yueyu1030/attrprompt","repo_kind":"official","path":"train_classifier/probing_model/sentence_bert.py","file_url":"https://github.com/yueyu1030/attrprompt/blob/HEAD/train_classifier/probing_model/sentence_bert.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"77f69af078e09830"}},{"code_sha256_prefix":"a4728a07cdbdb9de","entry":"split_list","repo":"yueyu1030/AttrPrompt","repo_kind":"official","path":"gen_train_data/filter.py","file_url":"https://github.com/yueyu1030/AttrPrompt/blob/HEAD/gen_train_data/filter.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a4728a07cdbdb9de"}},{"code_sha256_prefix":"90368409bfec83d7","entry":"call_api_async","repo":"yueyu1030/attrprompt","repo_kind":"official","path":"gen_train_data/gpt_gen_attrprompt.py","file_url":"https://github.com/yueyu1030/attrprompt/blob/HEAD/gen_train_data/gpt_gen_attrprompt.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"90368409bfec83d7"}},{"code_sha256_prefix":"4f6fe8fe6149448b","entry":"call_api_async","repo":"yueyu1030/attrprompt","repo_kind":"official","path":"gen_train_data/gpt_gen_attr.py","file_url":"https://github.com/yueyu1030/attrprompt/blob/HEAD/gen_train_data/gpt_gen_attr.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4f6fe8fe6149448b"}},{"code_sha256_prefix":"99a40c2d6e975af6","entry":"clean_doc","repo":"yueyu1030/attrprompt","repo_kind":"official","path":"train_classifier/plm_model/utils.py","file_url":"https://github.com/yueyu1030/attrprompt/blob/HEAD/train_classifier/plm_model/utils.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"99a40c2d6e975af6"}},{"code_sha256_prefix":"585a6a66de240121","entry":"dispatch_openai_requests","repo":"yueyu1030/attrprompt","repo_kind":"official","path":"gen_train_data/gpt_gen_attrprompt.py","file_url":"https://github.com/yueyu1030/attrprompt/blob/HEAD/gen_train_data/gpt_gen_attrprompt.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"585a6a66de240121"}},{"code_sha256_prefix":"946fc808830ab300","entry":"load_attributes","repo":"yueyu1030/attrprompt","repo_kind":"official","path":"gen_train_data/utils.py","file_url":"https://github.com/yueyu1030/attrprompt/blob/HEAD/gen_train_data/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"946fc808830ab300"}},{"code_sha256_prefix":"ae390922a4923fec","entry":"load_class_name","repo":"yueyu1030/attrprompt","repo_kind":"official","path":"gen_train_data/merge_subtopics_multi_label.py","file_url":"https://github.com/yueyu1030/attrprompt/blob/HEAD/gen_train_data/merge_subtopics_multi_label.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ae390922a4923fec"}},{"code_sha256_prefix":"5926d28ad8e9b9c8","entry":"load_sentences","repo":"yueyu1030/attrprompt","repo_kind":"official","path":"train_classifier/probing_model/sentence_bert.py","file_url":"https://github.com/yueyu1030/attrprompt/blob/HEAD/train_classifier/probing_model/sentence_bert.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5926d28ad8e9b9c8"}},{"code_sha256_prefix":"8b6fa84f7cf60755","entry":"remove_numbered_lists","repo":"yueyu1030/attrprompt","repo_kind":"official","path":"gen_train_data/utils.py","file_url":"https://github.com/yueyu1030/attrprompt/blob/HEAD/gen_train_data/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8b6fa84f7cf60755"}},{"code_sha256_prefix":"a72878cb18852ea0","entry":"tokenization","repo":"yueyu1030/attrprompt","repo_kind":"official","path":"train_classifier/probing_model/sentence_bert.py","file_url":"https://github.com/yueyu1030/attrprompt/blob/HEAD/train_classifier/probing_model/sentence_bert.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a72878cb18852ea0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}