Dataset Distillation

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

No dataset record in the archive lists this task.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 120 papers with code (216 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Dataset Distillation by Matching Training Trajectories
  22 Mar 2022 6 repositories listed
  To efficiently obtain the initial and target network parameters for large-scale datasets, we pre-compute and store training trajectories of expert networks trained on the real dataset.
• Dataset Distillation
  27 Nov 2018 6 repositories listed Syntology ran 3 of 17 samples · 14 unverified · 3 pointer-only (licence)
  Model distillation aims to distill the knowledge of a complex model into a simpler one.
• On the Diversity and Realism of Distilled Dataset: An Efficient Dataset Distillation Paradigm
  6 Dec 2023 4 repositories listed Syntology ran 3 of 10 samples · 7 unverified · 4 pointer-only (licence)
  Contemporary machine learning requires training large neural networks on massive datasets and thus faces the challenges of high computational demands.
• FedCache 2.0: Federated Edge Learning with Knowledge Caching and Dataset Distillation
  22 May 2024 3 repositories listed
  FedCache 2.
• Minimizing the Accumulated Trajectory Error to Improve Dataset Distillation
  20 Nov 2022 3 repositories listed
  To mitigate the adverse impact of this accumulated trajectory error, we propose a novel approach that encourages the optimization algorithm to seek a flat trajectory.
• Dataset Distillation via Factorization
  30 Oct 2022 3 repositories listed Syntology ran 0 of 7 samples · 7 unverified
  In this paper, we study \xw{dataset distillation (DD)}, from a novel perspective and introduce a \emph{dataset factorization} approach, termed \emph{HaBa}, which is a plug-and-play strategy portable to any existing DD…
• Dataset Distillation with Infinitely Wide Convolutional Networks
  27 Jul 2021 3 repositories listed
  The effectiveness of machine learning algorithms arises from being able to extract useful features from large amounts of data.
• Soft-Label Dataset Distillation and Text Dataset Distillation
  6 Oct 2019 3 repositories listed Syntology ran 0 of 13 samples · 13 unverified
  We propose to simultaneously distill both images and their labels, thus assigning each synthetic sample a `soft' label (a distribution of labels).
• Dataset Distillation via Vision-Language Category Prototype
  30 Jun 2025 2 repositories listed Syntology ran 1 of 2 samples · 1 unverified · 2 pointer-only (licence)
  In this study, we integrate vision-language methods into DD by introducing text prototypes to distill language information and collaboratively synthesize data with image prototypes, thereby enhancing dataset…
• Generative Dataset Distillation Based on Diffusion Model
  16 Aug 2024 2 repositories listed Syntology ran 2 of 2 samples · 0 unverified
  In this study, we proposed a novel generative dataset distillation method based on Stable Diffusion.
• Embarassingly Simple Dataset Distillation
  13 Nov 2023 2 repositories listed Syntology ran 1 of 1 samples · 0 unverified
  Re-examining the foundational back-propagation through time method, we study the pronounced variance in the gradients, computational burden, and long-term dependencies.
• Does Graph Distillation See Like Vision Dataset Counterpart?
  13 Oct 2023 2 repositories listed Syntology ran 8 of 9 samples · 1 unverified
  We validate the proposed SGDD across 9 datasets and achieve state-of-the-art results on all of them: for example, on the YelpChi dataset, our approach maintains 98.
• Self-Supervised Dataset Distillation for Transfer Learning
  10 Oct 2023 2 repositories listed Syntology ran 2 of 3 samples · 1 unverified
  To achieve this, we also introduce the MSE between representations of the inner model and the self-supervised target model on the original full dataset for outer optimization.
• DataDAM: Efficient Dataset Distillation with Attention Matching
  29 Sep 2023 2 repositories listed Syntology ran 35 of 43 samples · 8 unverified · 43 pointer-only (licence)
  Emerging research on dataset distillation aims to reduce training costs by creating a small synthetic set that contains the information of a larger real dataset and ultimately achieves test accuracy equivalent to a…
• Vision-Language Dataset Distillation
  15 Aug 2023 2 repositories listed Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
  In this work, we design the first vision-language dataset distillation method, building on the idea of trajectory matching.
• Towards Trustworthy Dataset Distillation
  18 Jul 2023 2 repositories listed
  By distilling both InD samples and outliers, the condensed datasets are capable of training models competent in both InD classification and OOD detection.
• Squeeze, Recover and Relabel: Dataset Condensation at ImageNet Scale From A New Perspective
  22 Jun 2023 2 repositories listed Syntology ran 1 of 6 samples · 5 unverified · 6 pointer-only (licence)
  The proposed method demonstrates flexibility across diverse dataset scales and exhibits multiple advantages in terms of arbitrary resolutions of synthesized images, low training cost and memory consumption with…
• Distill Gold from Massive Ores: Bi-level Data Pruning towards Efficient Dataset Distillation
  28 May 2023 2 repositories listed Syntology ran 1 of 8 samples · 7 unverified
  We believe this paradigm will open up new avenues in the dynamics of distillation and pave the way for efficient dataset distillation.
• Generalizing Dataset Distillation via Deep Generative Prior
  2 May 2023 2 repositories listed
  Dataset Distillation aims to distill an entire dataset's knowledge into a few synthetic images.
• DiM: Distilling Dataset into Generative Model
  8 Mar 2023 2 repositories listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  To the best of our knowledge, we are the first to achieve higher accuracy on complex architectures than simple ones, such as 75.
• DREAM: Efficient Dataset Distillation by Representative Matching
  28 Feb 2023 2 repositories listed
  Although there are various matching objectives, currently the strategy for selecting original images is limited to naive random sampling.
• Dataset Distillation with Convexified Implicit Gradients
  13 Feb 2023 2 repositories listed
  We propose a new dataset distillation algorithm using reparameterization and convexification of implicit gradients (RCIG), that substantially improves the state-of-the-art.
• Backdoor Attacks Against Dataset Distillation
  3 Jan 2023 2 repositories listed Syntology ran 6 of 15 samples · 9 unverified
  A model trained on this smaller distilled dataset can attain comparable performance to a model trained on the original training dataset.
• Accelerating Dataset Distillation via Model Augmentation
  12 Dec 2022 2 repositories listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  Dataset Distillation (DD), a newly emerging field, aims at generating much smaller but efficient synthetic training datasets from large ones.
• Scaling Up Dataset Distillation to ImageNet-1K with Constant Memory
  19 Nov 2022 2 repositories listed Syntology ran 0 of 2 samples · 2 unverified
  The resulting algorithm sets new SOTA on ImageNet-1K: we can scale up to 50 IPCs (Image Per Class) on ImageNet-1K on a single GPU (all previous methods can only scale to 2 IPCs on ImageNet-1K), leading to the best…
• Efficient Dataset Distillation Using Random Feature Approximation
  21 Oct 2022 2 repositories listed Syntology ran 1 of 2 samples · 1 unverified · 2 pointer-only (licence)
  Dataset distillation compresses large datasets into smaller synthetic coresets which retain performance with the aim of reducing the storage and computational burden of processing the entire dataset.
• Federated Learning via Decentralized Dataset Distillation in Resource-Constrained Edge Environments
  24 Aug 2022 2 repositories listed Syntology ran 1 of 5 samples · 4 unverified
  In federated learning, all networked clients contribute to the model training cooperatively.
• Remember the Past: Distilling Datasets into Addressable Memories for Neural Networks
  6 Jun 2022 2 repositories listed
  We propose an algorithm that compresses the critical information of a large dataset into compact addressable memories.
• Dataset Distillation using Neural Feature Regression
  1 Jun 2022 2 repositories listed Syntology ran 0 of 1 samples · 1 unverified · 1 pointer-only (licence)
  Dataset distillation can be formulated as a bi-level meta-learning problem where the outer loop optimizes the meta-dataset and the inner loop trains a model on the distilled data.
• Flexible Dataset Distillation: Learn Labels Instead of Images
  15 Jun 2020 2 repositories listed Syntology ran 12 of 14 samples · 2 unverified
  In particular, we study the problem of label distillation - creating synthetic labels for a small set of real images, and show it to be more effective than the prior image-based approach to dataset distillation.

Syntology lines on 21 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.