Browse State-of-the-Art › Knowledge Distillation › Papers, page 36

Knowledge Distillation

Papers archive 2025-07-28

archive papers tagged: 4,240 · with a code link: 1,740 · where Syntology ran a sample: 451 (380 with a run with no instrument failure, 71 where every run was a failure of Syntology's instrument) Syntology

Show: all tagged papersonly where code ran (451 of 4,240 tagged: 380 with a run with no instrument failure, 71 where every run was a failure of Syntology's instrument)

Page 36 of 43: papers 3,501 to 3,600 of 4,240, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.

Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. Abstracts are on each paper's page.

• Handling Long-tailed Feature Distribution in AdderNets
  1 Dec 2021 0 repositories listed
• Unsupervised Representation Transfer for Small Networks: I Believe I Can Distill On-the-Fly
  1 Dec 2021 0 repositories listed
• Using a GAN to Generate Adversarial Examples to Facial Image Recognition
  30 Nov 2021 0 repositories listed
• Efficient Federated Learning for AIoT Applications Using Knowledge Distillation
  29 Nov 2021 0 repositories listed
• Improved Knowledge Distillation via Adversarial Collaboration
  29 Nov 2021 0 repositories listed
• ESGN: Efficient Stereo Geometry Network for Fast 3D Object Detection
  28 Nov 2021 0 repositories listed
• Ensembling of Distilled Models from Multi-task Teachers for Constrained Resource Language Pairs
  26 Nov 2021 0 repositories listed
• Domain-Agnostic Clustering with Self-Distillation
  23 Nov 2021 0 repositories listed
• Contrast-reconstruction Representation Learning for Self-supervised Skeleton-based Action Recognition
  22 Nov 2021 0 repositories listed
• Hierarchical Knowledge Distillation for Dialogue Sequence Labeling
  22 Nov 2021 0 repositories listed
• Local-Selective Feature Distillation for Single Image Super-Resolution
  22 Nov 2021 0 repositories listed
• Teacher-Student Training and Triplet Loss to Reduce the Effect of Drastic Face Occlusion
  20 Nov 2021 0 repositories listed
• Toxicity Detection can be Sensitive to the Conversational Context
  19 Nov 2021 0 repositories listed
• Dynamically pruning segformer for efficient semantic segmentation
  18 Nov 2021 0 repositories listed
• Hierarchical Knowledge Guided Learning for Real-world Retinal Diseases Recognition
  17 Nov 2021 0 repositories listed
• A Flexible Multi-Task Model for BERT Serving
  16 Nov 2021 0 repositories listed
• Aligned Weight Regularizers for Pruning Pretrained Neural Networks
  16 Nov 2021 0 repositories listed
• Compositional Data Augmentation for Abstractive Conversation Summarization
  16 Nov 2021 0 repositories listed
• Deep-to-bottom Weights Decay: A Systemic Knowledge Review Learning Technique for Transformer Layers in Knowledge Distillation
  16 Nov 2021 0 repositories listed
• Enabling Multimodal Generation on CLIP via Vision-Language Knowledge Distillation
  16 Nov 2021 0 repositories listed
• Feature Structure Distillation for BERT Transferring
  16 Nov 2021 0 repositories listed
• Learning to Teach with Student Feedback
  16 Nov 2021 0 repositories listed
• Making Small Language Models Better Few-Shot Learners
  16 Nov 2021 0 repositories listed
• Multi-Granularity Contrastive Knowledge Distillation for Multimodal Named Entity Recognition
  16 Nov 2021 0 repositories listed
• Multi-stage Distillation Framework for Cross-Lingual Semantic Similarity Matching
  16 Nov 2021 0 repositories listed
• NVIDIA NeMo Neural Machine Translation Systems for English-German and English-Russian News and Biomedical Tasks at WMT21
  16 Nov 2021 0 repositories listed
• One General Teacher for Multi-Data Multi-Task: A New Knowledge Distillation Framework for Discourse Relation Analysis
  16 Nov 2021 0 repositories listed
• Redistributing Low-Frequency Words: Making the Most of Monolingual Data in Non-Autoregressive Translation
  16 Nov 2021 0 repositories listed
• Self-Distilled Pruning of Neural Networks
  16 Nov 2021 0 repositories listed
• Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm
  16 Nov 2021 0 repositories listed
• When Chosen Wisely, More Data Is What You Need: A Universal Sample-Efficient Strategy For Data Augmentation
  16 Nov 2021 0 repositories listed
• Synthetic Unknown Class Learning for Learning Unknowns
  15 Nov 2021 0 repositories listed
• Domain Generalization on Efficient Acoustic Scene Classification using Residual Normalization
  12 Nov 2021 0 repositories listed
• Learning Interpretation with Explainable Knowledge Distillation
  12 Nov 2021 0 repositories listed
• A Survey on Green Deep Learning
  8 Nov 2021 0 repositories listed
• Class Token and Knowledge Distillation for Multi-head Self-Attention Speaker Verification Systems
  6 Nov 2021 0 repositories listed
• AUTOKD: Automatic Knowledge Distillation Into A Student Architecture Family
  5 Nov 2021 0 repositories listed
• DVFL: A Vertical Federated Learning Method for Dynamic Data
  5 Nov 2021 0 repositories listed
• A methodology for training homomorphicencryption friendly neural networks
  5 Nov 2021 0 repositories listed
• Oracle Teacher: Leveraging Target Information for Better Knowledge Distillation of CTC Models
  5 Nov 2021 0 repositories listed
• Visualizing the Emergence of Intermediate Visual Patterns in DNNs
  5 Nov 2021 0 repositories listed
• Leveraging Advantages of Interactive and Non-Interactive Models for Vector-Based Cross-Lingual Information Retrieval
  3 Nov 2021 0 repositories listed
• Knowledge Cross-Distillation for Membership Privacy
  2 Nov 2021 0 repositories listed
• AUTOSUMM: Automatic Model Creation for Text Summarization
  1 Nov 2021 0 repositories listed
• Combining Curriculum Learning and Knowledge Distillation for Dialogue Generation
  1 Nov 2021 0 repositories listed
• GAML-BERT: Improving BERT Early Exiting by Gradient Aligned Mutual Learning
  1 Nov 2021 0 repositories listed
• How to Select One Among All ? An Empirical Study Towards the Robustness of Knowledge Distillation in Natural Language Understanding
  1 Nov 2021 0 repositories listed
• HW-TSC’s Participation in the WMT 2021 Large-Scale Multilingual Translation Task
  1 Nov 2021 0 repositories listed
• HW-TSC’s Participation in the WMT 2021 News Translation Shared Task
  1 Nov 2021 0 repositories listed
• Limitations of Knowledge Distillation for Zero-shot Transfer Learning
  1 Nov 2021 0 repositories listed
• Multilingual Neural Machine Translation: Can Linguistic Hierarchies Help?
  1 Nov 2021 0 repositories listed
• Mutual-Learning Improves End-to-End Speech Translation
  1 Nov 2021 0 repositories listed
• NVIDIA NeMo’s Neural Machine Translation Systems for English-German and English-Russian News and Biomedical Tasks at WMT21
  1 Nov 2021 0 repositories listed
• Papago’s Submission for the WMT21 Quality Estimation Shared Task
  1 Nov 2021 0 repositories listed
• PDALN: Progressive Domain Adaptation over a Pre-trained Model for Low-Resource Cross-Domain Named Entity Recognition
  1 Nov 2021 0 repositories listed
• RW-KD: Sample-wise Loss Terms Re-Weighting for Knowledge Distillation
  1 Nov 2021 0 repositories listed
• Students Who Study Together Learn Better: On the Importance of Collective Knowledge Distillation for Domain Transfer in Fact Verification
  1 Nov 2021 0 repositories listed
• TenTrans Large-Scale Multilingual Machine Translation System for WMT21
  1 Nov 2021 0 repositories listed
• The LMU Munich System for the WMT 2021 Large-Scale Multilingual Machine Translation Shared Task
  1 Nov 2021 0 repositories listed
• The Mininglamp Machine Translation System for WMT21
  1 Nov 2021 0 repositories listed
• The NiuTrans System for the WMT 2021 Efficiency Task
  1 Nov 2021 0 repositories listed
• Universal-KD: Attention-based Output-Grounded Intermediate Layer Knowledge Distillation
  1 Nov 2021 0 repositories listed
• Rethinking the Knowledge Distillation From the Perspective of Model Calibration
  31 Oct 2021 0 repositories listed
• Estimating and Maximizing Mutual Information for Knowledge Distillation
  29 Oct 2021 0 repositories listed
• On Cross-Layer Alignment for Model Fusion of Heterogeneous Neural Networks
  29 Oct 2021 0 repositories listed
• NxMTransformer: Semi-Structured Sparsification for Natural Language Understanding via ADMM
  28 Oct 2021 0 repositories listed
• Towards Model Agnostic Federated Learning Using Knowledge Distillation
  28 Oct 2021 0 repositories listed
• Beyond Classification: Knowledge Distillation using Multi-Object Impressions
  27 Oct 2021 0 repositories listed
• Temporal Knowledge Distillation for On-device Audio Classification
  27 Oct 2021 0 repositories listed
• Response-based Distillation for Incremental Object Detection
  26 Oct 2021 0 repositories listed
• MUSE: Feature Self-Distillation with Mutual Information and Self-Information
  25 Oct 2021 0 repositories listed
• Reconstructing Pruned Filters using Cheap Spatial Transformations
  25 Oct 2021 0 repositories listed
• X-Distill: Improving Self-Supervised Monocular Depth via Cross-Task Distillation
  24 Oct 2021 0 repositories listed
• How and When Adversarial Robustness Transfers in Knowledge Distillation?
  22 Oct 2021 0 repositories listed
• Pseudo Supervised Monocular Depth Estimation with Teacher-Student Network
  22 Oct 2021 0 repositories listed
• Augmenting Knowledge Distillation With Peer-To-Peer Mutual Learning For Model Compression
  21 Oct 2021 0 repositories listed
• Class Incremental Online Streaming Learning
  20 Oct 2021 0 repositories listed
• Knowledge distillation from language model to acoustic model: a hierarchical multi-task learning approach
  20 Oct 2021 0 repositories listed
• A Short Study on Compressing Decoder-Based Language Models
  16 Oct 2021 0 repositories listed
• Know your tools well: Better and faster QA with synthetic examples
  16 Oct 2021 0 repositories listed
• Pro-KD: Progressive Distillation by Following the Footsteps of the Teacher
  16 Oct 2021 0 repositories listed
• Robustness Challenges in Model Distillation and Pruning for Natural Language Understanding
  16 Oct 2021 0 repositories listed
• From Multimodal to Unimodal Attention in Transformers using Knowledge Distillation
  15 Oct 2021 0 repositories listed
• Kronecker Decomposition for GPT Compression
  15 Oct 2021 0 repositories listed
• Multilingual Neural Machine Translation:Can Linguistic Hierarchies Help?
  15 Oct 2021 0 repositories listed
• Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm
  15 Oct 2021 0 repositories listed
• False Negative Distillation and Contrastive Learning for Personalized Outfit Recommendation
  13 Oct 2021 0 repositories listed
• Language Modelling via Learning to Rank
  13 Oct 2021 0 repositories listed
• Compact CNN Models for On-device Ocular-based User Recognition in Mobile Devices
  11 Oct 2021 0 repositories listed
• Rectifying the Data Bias in Knowledge Distillation
  11 Oct 2021 0 repositories listed
• Towards Streaming Egocentric Action Anticipation
  11 Oct 2021 0 repositories listed
• Visualizing the embedding space to explain the effect of knowledge distillation
  9 Oct 2021 0 repositories listed
• Knowledge Distillation for Neural Transducers from Large Self-Supervised Pre-trained Models
  7 Oct 2021 0 repositories listed
• Peer Collaborative Learning for Polyphonic Sound Event Detection
  7 Oct 2021 0 repositories listed
• Online Hyperparameter Meta-Learning with Hypergradient Distillation
  6 Oct 2021 0 repositories listed
• On the Interplay Between Sparsity, Naturalness, Intelligibility, and Prosody in Speech Synthesis
  4 Oct 2021 0 repositories listed
• Student Helping Teacher: Teacher Evolution via Self-Knowledge Distillation
  1 Oct 2021 0 repositories listed
• Improving Neural Ranking via Lossless Knowledge Distillation
  30 Sep 2021 0 repositories listed
• Deep Neural Compression Via Concurrent Pruning and Self-Distillation
  30 Sep 2021 0 repositories listed
• A Comprehensive Overhaul of Distilling Unconditional GANs
  29 Sep 2021 0 repositories listed