{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/synthesizrr-generating-diverse-datasets-with","title":"SynthesizRR: Generating Diverse Datasets with Retrieval Augmentation","arxiv_id":"2405.10040","date":"2024-05-16","proceeding":null,"authors":["Abhishek Divekar","Greg Durrett"],"abstract":"It is often desirable to distill the capabilities of large language models (LLMs) into smaller student models due to compute and memory constraints. One way to do this for classification tasks is via dataset synthesis, which can be accomplished by generating examples of each label from the LLM. Prior approaches to synthesis use few-shot prompting, which relies on the LLM's parametric knowledge to generate usable examples. However, this leads to issues of repetition, bias towards popular entities, and stylistic differences from human text. In this work, we propose Synthesize by Retrieval and Refinement (SynthesizRR), which uses retrieval augmentation to introduce variety into the dataset synthesis process: as retrieved passages vary, the LLM is seeded with different content to generate its examples. We empirically study the synthesis of six datasets, covering topic classification, sentiment analysis, tone detection, and humor, requiring complex synthesis strategies. We find that SynthesizRR greatly improves lexical and semantic diversity, similarity to human-written text, and distillation performance, when compared to 32-shot prompting and four prior approaches. We release our code to perform all steps at https://github.com/amazon-science/synthesizrr","url_abs":"https://arxiv.org/abs/2405.10040v3","url_pdf":"https://arxiv.org/pdf/2405.10040v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"synthesizrr-generating-diverse-datasets-with","repo_url":"https://github.com/amazon-science/synthesizrr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"bias-detection","task_name":"Bias Detection"},{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"humor-detection","task_name":"Humor Detection"},{"task_slug":"information-retrieval","task_name":"Information Retrieval"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"product-categorization","task_name":"Product Categorization"},{"task_slug":"retrieval","task_name":"Retrieval"},{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"},{"task_slug":"synthetic-data-generation","task_name":"Synthetic Data Generation"},{"task_slug":"topic-classification","task_name":"Topic Classification"}],"methods":[{"method_slug":"synthesizrr","method_name":"SynthesizRR"}],"datasets_introduced":[],"methods_introduced":[{"slug":"synthesizrr","name":"SynthesizRR","full_name":"Synthesize by Retrieval and Refinement"}],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2405.10040","atlas_url":"https://app.syntology.ai/?focus=2405.10040","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.10040"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/amazon-science/synthesizrr","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":1},"by_repo_kind":{"official":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"4f71a25c60a38609","entry":"count_num_tokens","repo":"amazon-science/synthesizrr","repo_kind":"official","path":"src/synthesizrr/corpus.py","file_url":"https://github.com/amazon-science/synthesizrr/blob/HEAD/src/synthesizrr/corpus.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4f71a25c60a38609"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}