Methods › General › Knowledge Distillation › Knowledge Distillation › Papers, page 24
Knowledge Distillation
Papers archive 2025-07-28
archive papers tagged: 3,071 · with a code link: 1,258 · where Syntology ran a sample: 320 (276 with a run with no instrument failure, 44 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (320 of 3,071 tagged: 276 with a run with no instrument failure, 44 where every run was a failure of Syntology's instrument)
Page 24 of 31: papers 2,301 to 2,400 of 3,071, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
Adaptive Distillation: Aggregating Knowledge from Multiple Paths for Efficient Distillation 19 Oct 2021 · 1 repository · arXiv:2110.09674
-
Graph-less Neural Networks: Teaching Old MLPs New Tricks via Distillation 17 Oct 2021 · 1 repository · arXiv:2110.08727Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 1 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 8 harvested samples) · 8 pointer-only (licence)
-
A Short Study on Compressing Decoder-Based Language Models 16 Oct 2021 · 0 repositories · arXiv:2110.08460
-
HRKD: Hierarchical Relational Knowledge Distillation for Cross-domain Language Model Compression 16 Oct 2021 · 1 repository · arXiv:2110.08551Syntology official (archive's flag): 3 ran · 3 ran (of which 3 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 3 samples that ran constructed an object rather than computing a result (of 3 harvested samples) · 3 pointer-only (licence)
-
Pro-KD: Progressive Distillation by Following the Footsteps of the Teacher 16 Oct 2021 · 0 repositories · arXiv:2110.08532
-
Robustness Challenges in Model Distillation and Pruning for Natural Language Understanding 16 Oct 2021 · 0 repositories · arXiv:2110.08419
-
From Multimodal to Unimodal Attention in Transformers using Knowledge Distillation 15 Oct 2021 · 0 repositories · arXiv:2110.08270
-
Kronecker Decomposition for GPT Compression 15 Oct 2021 · 0 repositories · arXiv:2110.08152
-
Multilingual Neural Machine Translation:Can Linguistic Hierarchies Help? 15 Oct 2021 · 0 repositories · arXiv:2110.07816
-
Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm 15 Oct 2021 · 0 repositories · arXiv:2110.08190
-
FocusNet: Classifying Better by Focusing on Confusing Classes 14 Oct 2021 · 2 repositories · arXiv:2110.07307
-
Symbolic Knowledge Distillation: from General Language Models to Commonsense Models 14 Oct 2021 · 1 repository · arXiv:2110.07178
-
False Negative Distillation and Contrastive Learning for Personalized Outfit Recommendation 13 Oct 2021 · 0 repositories · arXiv:2110.06483
-
Language Modelling via Learning to Rank 13 Oct 2021 · 0 repositories · arXiv:2110.06961
-
Object DGCNN: 3D Object Detection using Dynamic Graphs 13 Oct 2021 · 1 repository · arXiv:2110.06923Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples) · 3 pointer-only (licence)
-
Compact CNN Models for On-device Ocular-based User Recognition in Mobile Devices 11 Oct 2021 · 0 repositories · arXiv:2110.04953
-
Towards Streaming Egocentric Action Anticipation 11 Oct 2021 · 0 repositories · arXiv:2110.05386
-
Towards Data-Free Domain Generalization 9 Oct 2021 · 1 repository · arXiv:2110.04545Syntology official (archive's flag): 15 ran · 15 ran (of which 5 constructed an object rather than computing a result; 7 with no instrument failure: 2 honoured, 0 violated, 5 with no contract checked; 8 where Syntology's instrument failed) · 5 unverified (of 20 harvested samples)
-
Visualizing the embedding space to explain the effect of knowledge distillation 9 Oct 2021 · 0 repositories · arXiv:2110.04483
-
Cross-modal Knowledge Distillation for Vision-to-Sensor Action Recognition 8 Oct 2021 · 1 repository · arXiv:2112.01849
-
Knowledge Distillation for Neural Transducers from Large Self-Supervised Pre-trained Models 7 Oct 2021 · 0 repositories · arXiv:2110.03334
-
Peer Collaborative Learning for Polyphonic Sound Event Detection 7 Oct 2021 · 0 repositories · arXiv:2110.03511
-
Towards Accurate Cross-Domain In-Bed Human Pose Estimation 7 Oct 2021 · 1 repository · arXiv:2110.03578
-
KNOT: Knowledge Distillation using Optimal Transport for Solving NLP Tasks 6 Oct 2021 · 2 repositories · arXiv:2110.02432
-
Inter-Domain Alignment for Predicting High-Resolution Brain Networks Using Teacher-Student Learning 6 Oct 2021 · 1 repository · arXiv:2110.03452
-
On the Interplay Between Sparsity, Naturalness, Intelligibility, and Prosody in Speech Synthesis 4 Oct 2021 · 0 repositories · arXiv:2110.01147
-
STEM: An Approach to Multi-Source Domain Adaptation With Guarantees 1 Oct 2021 · 1 repository
-
Improving Neural Ranking via Lossless Knowledge Distillation 30 Sep 2021 · 0 repositories · arXiv:2109.15285
-
Prune Your Model Before Distill It 30 Sep 2021 · 1 repository · arXiv:2109.14960
-
A Comprehensive Overhaul of Distilling Unconditional GANs 29 Sep 2021 · 0 repositories
-
A Unified Knowledge Distillation Framework for Deep Directed Graphical Models 29 Sep 2021 · 0 repositories
-
Automated Channel Pruning with Learned Importance 29 Sep 2021 · 0 repositories
-
Distilling GANs with Style-Mixed Triplets for X2I Translation with Limited Data 29 Sep 2021 · 0 repositories
-
Fast and Efficient Once-For-All Networks for Diverse Hardware Deployment 29 Sep 2021 · 0 repositories
-
Feature Kernel Distillation 29 Sep 2021 · 0 repositories
-
Generate, Annotate, and Learn: Generative Models Advance Self-Training and Knowledge Distillation 29 Sep 2021 · 0 repositories
-
MOBA: Multi-teacher Model Based Reinforcement Learning 29 Sep 2021 · 0 repositories
-
Neural Architecture Search via Ensemble-based Knowledge Distillation 29 Sep 2021 · 0 repositories
-
Not All Regions are Worthy to be Distilled: Region-aware Knowledge Distillation Towards Efficient Image-to-Image Translation 29 Sep 2021 · 0 repositories
-
Prototypical Contrastive Predictive Coding 29 Sep 2021 · 0 repositories
-
Reducing the Teacher-Student Gap via Adaptive Temperatures 29 Sep 2021 · 0 repositories
-
Representation Consolidation from Multiple Expert Teachers 29 Sep 2021 · 0 repositories
-
Scale-Invariant Teaching for Semi-Supervised Object Detection 29 Sep 2021 · 0 repositories
-
Self-Slimming Vision Transformer 29 Sep 2021 · 0 repositories
-
Self-supervised Models are Good Teaching Assistants for Vision Transformers 29 Sep 2021 · 0 repositories
-
SeqPATE: Differentially Private Text Generation via Knowledge Distillation 29 Sep 2021 · 0 repositories
-
Source-Target Unified Knowledge Distillation for Memory-Efficient Federated Domain Adaptation on Edge Devices 29 Sep 2021 · 0 repositories
-
To Smooth or not to Smooth? On Compatibility between Label Smoothing and Knowledge Distillation 29 Sep 2021 · 0 repositories
-
Wakening Past Concepts without Past Data: Class-incremental Learning from Placebos 29 Sep 2021 · 0 repositories
-
Improving Question Answering Performance Using Knowledge Distillation and Active Learning 26 Sep 2021 · 1 repository · arXiv:2109.12662Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples) · 3 pointer-only (licence)
-
Partial to Whole Knowledge Distillation: Progressive Distilling Decomposed Knowledge Boosts Student Better 26 Sep 2021 · 0 repositories · arXiv:2109.12507
-
Dynamic Knowledge Distillation for Pre-trained Language Models 23 Sep 2021 · 1 repository · arXiv:2109.11295
-
K-AID: Enhancing Pre-trained Language Models with Domain Knowledge for Question Answering 22 Sep 2021 · 0 repositories · arXiv:2109.10547
-
KD-VLP: Improving End-to-End Vision-and-Language Pretraining with Object Knowledge Distillation 22 Sep 2021 · 1 repository · arXiv:2109.10504
-
Low-Latency Incremental Text-to-Speech Synthesis with Distilled Context Prediction Network 22 Sep 2021 · 0 repositories · arXiv:2109.10724
-
RAIL-KD: RAndom Intermediate Layer Mapping for Knowledge Distillation 21 Sep 2021 · 0 repositories · arXiv:2109.10164
-
Distilling Linguistic Context for Language Model Compression 17 Sep 2021 · 1 repository · arXiv:2109.08359Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 0 violated, 7 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified (of 11 harvested samples)
-
Towards Full Utilization on Mask Task for Distilling PLMs into NMT 17 Sep 2021 · 0 repositories
-
Label Assignment Distillation for Object Detection 16 Sep 2021 · 0 repositories · arXiv:2109.07843
-
The NiuTrans System for the WMT21 Efficiency Task 16 Sep 2021 · 1 repository · arXiv:2109.08003
-
EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation 15 Sep 2021 · 1 repository · arXiv:2109.07222Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
New Perspective on Progressive GANs Distillation for One-class Novelty Detection 15 Sep 2021 · 0 repositories · arXiv:2109.07295
-
Multihop: Leveraging Complex Models to Learn Accurate Simple Models 14 Sep 2021 · 0 repositories · arXiv:2109.06961
-
A Note on Knowledge Distillation Loss Function for Object Classification 14 Sep 2021 · 0 repositories · arXiv:2109.06458
-
Multi-Scale Aligned Distillation for Low-Resolution Detection 14 Sep 2021 · 2 repositories · arXiv:2109.06875Syntology official (archive's flag): 4 ran · 4 ran (of which 1 constructed an object rather than computing a result; 4 with no instrument failure: 1 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 4 harvested samples) · 4 pointer-only (licence)
-
Secure Your Ride: Real-time Matching Success Rate Prediction for Passenger-Driver Pairs 14 Sep 2021 · 0 repositories · arXiv:2109.07571
-
KroneckerBERT: Learning Kronecker Decomposition for Pre-trained Language Models via Knowledge Distillation 13 Sep 2021 · 0 repositories · arXiv:2109.06243
-
UniMS: A Unified Framework for Multimodal Summarization with Knowledge Distillation 13 Sep 2021 · 0 repositories · arXiv:2109.05812
-
Federated Ensemble Model-based Reinforcement Learning in Edge Computing 12 Sep 2021 · 0 repositories · arXiv:2109.05549
-
On the Efficiency of Subclass Knowledge Distillation in Classification Tasks 12 Sep 2021 · 0 repositories · arXiv:2109.05587
-
Learning to Teach with Student Feedback 10 Sep 2021 · 0 repositories · arXiv:2109.04641
-
Towards Developing a Multilingual and Code-Mixed Visual Question Answering System by Knowledge Distillation 10 Sep 2021 · 0 repositories · arXiv:2109.04653
-
Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression 7 Sep 2021 · 1 repository · arXiv:2109.03228
-
Knowledge Distillation Using Hierarchical Self-Supervision Augmented Distribution 7 Sep 2021 · 1 repository · arXiv:2109.03075
-
CAM-loss: Towards Learning Spatially Discriminative Feature Representations 3 Sep 2021 · 0 repositories · arXiv:2109.01359
-
Catastrophic Interference in Reinforcement Learning: A Solution Based on Context Division and Knowledge Distillation 1 Sep 2021 · 1 repository · arXiv:2109.00525Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)
-
Decoupled Transformer for Scalable Inference in Open-domain Question Answering 1 Sep 2021 · 0 repositories
-
Knowledge Distillation with BERT for Image Tag-Based Privacy Prediction 1 Sep 2021 · 0 repositories
-
Lipschitz Continuity Guided Knowledge Distillation 29 Aug 2021 · 0 repositories · arXiv:2108.12905
-
Distilling the Knowledge of Large-scale Generative Models into Retrieval Models for Efficient Open-domain Conversation 28 Aug 2021 · 1 repository · arXiv:2108.12582
-
CoCo DistillNet: a Cross-layer Correlation Distillation Network for Pathological Gastric Cancer Segmentation 27 Aug 2021 · 0 repositories · arXiv:2108.12173
-
Efficient training of lightweight neural networks using Online Self-Acquired Knowledge Distillation 26 Aug 2021 · 0 repositories · arXiv:2108.11798
-
PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and Multi-Step Knowledge Distillation 24 Aug 2021 · 1 repository · arXiv:2108.10710
-
Deploying a BERT-based Query-Title Relevance Classifier in a Production System: a View from the Trenches 23 Aug 2021 · 0 repositories · arXiv:2108.10197
-
Supervised Compression for Resource-Constrained Edge Computing Systems 21 Aug 2021 · 2 repositories · arXiv:2108.11898Syntology official (archive's flag): 2 ran · 11 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 1 where Syntology's instrument failed) · 6 unverified (of 17 harvested samples) · 1 pointer-only (licence)
-
Boosting of Head Pose Estimation by Knowledge Distillation 20 Aug 2021 · 0 repositories · arXiv:2108.09183
-
Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Better 18 Aug 2021 · 1 repository · arXiv:2108.07969Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples)
-
BERT Learns to Teach: Knowledge Distillation with Meta Learning 17 Aug 2021 · 0 repositories
-
G-DetKD: Towards General Distillation Framework for Object Detectors via Contrastive and Semantic-guided Feature Imitation 17 Aug 2021 · 0 repositories · arXiv:2108.07482
-
Online Continual Learning For Visual Food Classification 15 Aug 2021 · 0 repositories · arXiv:2108.06781
-
AGKD-BML: Defense Against Adversarial Attack by Attention Guided Knowledge Distillation and Bi-directional Metric Learning 13 Aug 2021 · 1 repository · arXiv:2108.06017Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
Learning from Matured Dumb Teacher for Fine Generalization 12 Aug 2021 · 0 repositories · arXiv:2108.05776
-
Preventing Catastrophic Forgetting and Distribution Mismatch in Knowledge Distillation via Synthetic Data 11 Aug 2021 · 0 repositories · arXiv:2108.05698
-
Semi-Supervised Domain Generalizable Person Re-Identification 11 Aug 2021 · 3 repositories · arXiv:2108.05045Syntology official (archive's flag): 2 ran · 6 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples) · 3 pointer-only (licence)
-
Learning an Augmented RGB Representation with Cross-Modal Knowledge Distillation for Action Detection 8 Aug 2021 · 0 repositories · arXiv:2108.03619
-
A distillation based approach for the diagnosis of diseases 7 Aug 2021 · 0 repositories · arXiv:2108.03470
-
Spatio-Temporal Attention Mechanism and Knowledge Distillation for Lip Reading 7 Aug 2021 · 0 repositories · arXiv:2108.03543
-
Transferring Knowledge Distillation for Multilingual Social Event Detection 6 Aug 2021 · 2 repositories · arXiv:2108.03084
-
Decoupled Transformer for Scalable Inference in Open-domain Question Answering 5 Aug 2021 · 0 repositories · arXiv:2108.02765
-
Knowledge Distillation from BERT Transformer to Speech Transformer for Intent Classification 5 Aug 2021 · 1 repository · arXiv:2108.02598