{"url":"/method/synthesizrr","slug":"synthesizrr","name":"SynthesizRR","full_name":"Synthesize by Retrieval and Refinement","full_name_withheld":false,"description_markdown":"It is often desirable to distill the capabilities of large language models (LLMs) into smaller student models due to compute and memory constraints. One way to do this for classification tasks is via dataset synthesis. Prior approaches to synthesis use few-shot prompting, which relies on the LLM's parametric knowledge to generate usable examples. However, this leads to issues of repetition, bias towards popular entities, and stylistic differences from human text. We propose Synthesize by Retrieval and Refinement (SynthesizRR), which uses retrieval augmentation to introduce variety into the dataset synthesis process: as retrieved passages vary, the LLM is seeded with different content to generate its examples. We find that SynthesizRR greatly improves lexical and semantic diversity, similarity to human-written text, and distillation performance,","description_state":"present","introduced_year":null,"introduced_by":{"title":"SynthesizRR: Generating Diverse Datasets with Retrieval Augmentation","paper":"/paper/synthesizrr-generating-diverse-datasets-with","first_author":"Abhishek Divekar","n_authors":2,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/synthesizrr-generating-diverse-datasets-with"},"source":{"url":"https://arxiv.org/abs/2405.10040v3","title":"SynthesizRR: Generating Diverse Datasets with Retrieval Augmentation","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Natural Language Processing","area_id":"natural-language-processing","collection":"Text Data Augmentation","url":"/methods/category/text-data-augmentation","pwc_aliases":[]}],"n_papers_tagged":1,"archive_num_papers":1,"papers_newest_first":[{"paper":"/paper/synthesizrr-generating-diverse-datasets-with","title":"SynthesizRR: Generating Diverse Datasets with Retrieval Augmentation","date":"2024-05-16","arxiv_id":"2405.10040","n_code_links":1,"syntology":{"ran":0,"of":1,"unverified":1,"pointer_only":0}}],"papers_shown":1,"tasks":[{"task":"/task/bias-detection","name":"Bias Detection","papers":1},{"task":"/task/diversity","name":"Diversity","papers":1},{"task":"/task/humor-detection","name":"Humor Detection","papers":1},{"task":"/task/information-retrieval","name":"Information Retrieval","papers":1},{"task":"/task/language-modelling","name":"Language Modelling","papers":1},{"task":"/task/product-categorization","name":"Product Categorization","papers":1},{"task":"/task/retrieval","name":"Retrieval","papers":1},{"task":"/task/sentiment-analysis","name":"Sentiment Analysis","papers":1},{"task":"/task/synthetic-data-generation","name":"Synthetic Data Generation","papers":1},{"task":"/task/topic-classification","name":"Topic Classification","papers":1}],"tasks_shown":10,"n_tasks":10,"usage_by_year":[{"year":"2024","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/synthesizrr"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}