Methods › General › Knowledge Distillation › Knowledge Distillation › Papers, page 30

Knowledge Distillation

Papers archive 2025-07-28

archive papers tagged: 3,071 · with a code link: 1,258 · where Syntology ran a sample: 320 (276 with a run with no instrument failure, 44 where every run was a failure of Syntology's instrument) Syntology

Show: all tagged papersonly where code ran (320 of 3,071 tagged: 276 with a run with no instrument failure, 44 where every run was a failure of Syntology's instrument)

Page 30 of 31: papers 2,901 to 3,000 of 3,071, newest first by the archive's date (ties by slug), in archive order.

Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.

• Acquiring Knowledge from Pre-trained Model to Neural Machine Translation
  4 Dec 2019 · 0 repositories · arXiv:1912.01774
• QUEST: Quantized embedding space for transferring knowledge
  3 Dec 2019 · 1 repository · arXiv:1912.01540
• Efficient Convolutional Neural Networks for Depth-Based Multi-Person Pose Estimation
  2 Dec 2019 · 0 repositories · arXiv:1912.00711
• Online Knowledge Distillation with Diverse Peers
  1 Dec 2019 · 2 repositories · arXiv:1912.00350
• Random Path Selection for Continual Learning
  1 Dec 2019 · 1 repository
• Towards Oracle Knowledge Distillation with Neural Architecture Search
  29 Nov 2019 · 0 repositories · arXiv:1911.13019
• Data-Driven Compression of Convolutional Neural Networks
  28 Nov 2019 · 0 repositories · arXiv:1911.12740
• QKD: Quantization-aware Knowledge Distillation
  28 Nov 2019 · 0 repositories · arXiv:1911.12491
• Go From the General to the Particular: Multi-Domain Translation with Domain Transformation Networks
  22 Nov 2019 · 2 repositories · arXiv:1911.09912
• Few Shot Network Compression via Cross Distillation
  21 Nov 2019 · 1 repository · arXiv:1911.09450
• Search to Distill: Pearls are Everywhere but not the Eyes
  20 Nov 2019 · 0 repositories · arXiv:1911.09074
• Neural Network Pruning with Residual-Connections and Limited-Data
  19 Nov 2019 · 1 repository · arXiv:1911.08114
  Syntology 4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 3 unverified (of 7 harvested samples) · 2 pointer-only (licence)
• Maintaining Discrimination and Fairness in Class Incremental Learning
  16 Nov 2019 · 2 repositories · arXiv:1911.07053
• Data Efficient Stagewise Knowledge Distillation
  15 Nov 2019 · 1 repository · arXiv:1911.06786
• Knowledge Representing: Efficient, Sparse Representation of Prior Knowledge for Knowledge Distillation
  13 Nov 2019 · 0 repositories · arXiv:1911.05329
• Learning from a Teacher using Unlabeled Data
  13 Nov 2019 · 1 repository · arXiv:1911.05275
• Graph Representation Learning via Multi-task Knowledge Distillation
  11 Nov 2019 · 0 repositories · arXiv:1911.05700
• Knowledge Distillation in Document Retrieval
  11 Nov 2019 · 0 repositories · arXiv:1911.11065
• MKD: a Multi-Task Knowledge Distillation Approach for Pretrained Language Models
  9 Nov 2019 · 0 repositories · arXiv:1911.03588
• Deep geometric knowledge distillation with graphs
  8 Nov 2019 · 1 repository · arXiv:1911.03080
• Knowledge Distillation for Incremental Learning in Semantic Segmentation
  8 Nov 2019 · 0 repositories · arXiv:1911.03462
• Understanding Knowledge Distillation in Non-autoregressive Machine Translation
  7 Nov 2019 · 0 repositories · arXiv:1911.02727
• Data Diversification: A Simple Strategy For Neural Machine Translation
  5 Nov 2019 · 2 repositories · arXiv:1911.01986
  Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples) · 4 pointer-only (licence)
• Distilling Pixel-Wise Feature Similarities for Semantic Segmentation
  31 Oct 2019 · 0 repositories · arXiv:1910.14226
• A Simple but Effective BERT Model for Dialog State Tracking on Resource-Limited Systems
  28 Oct 2019 · 0 repositories · arXiv:1910.12995
• MOD: A Deep Mixture Model with Online Knowledge Distillation for Large Scale Video Temporal Concept Localization
  27 Oct 2019 · 1 repository · arXiv:1910.12295
• Variational Student: Learning Compact and Sparser Networks in Knowledge Distillation Framework
  26 Oct 2019 · 0 repositories · arXiv:1910.12061
• Adversarial Feature Alignment: Avoid Catastrophic Forgetting in Incremental Task Lifelong Learning
  24 Oct 2019 · 0 repositories · arXiv:1910.10986
• Contrastive Representation Distillation
  23 Oct 2019 · 4 repositories · arXiv:1910.10699
  Syntology official: no sample here; runs from other or unrecorded repositories · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
• Model Compression with Two-stage Multi-teacher Knowledge Distillation for Web Question Answering System
  18 Oct 2019 · 0 repositories · arXiv:1910.08381
• A Generalized and Robust Method Towards Practical Gaze Estimation on Smart Phone
  16 Oct 2019 · 0 repositories · arXiv:1910.07331
• Noise as a Resource for Learning in Knowledge Distillation
  11 Oct 2019 · 0 repositories · arXiv:1910.05057
• VarGFaceNet: An Efficient Variable Group Convolutional Neural Network for Lightweight Face Recognition
  11 Oct 2019 · 3 repositories · arXiv:1910.04985
• Cross-modal knowledge distillation for action recognition
  10 Oct 2019 · 0 repositories · arXiv:1910.04641
• FedMD: Heterogenous Federated Learning via Model Distillation
  8 Oct 2019 · 7 repositories · arXiv:1910.03581
  Syntology 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
• Knowledge Distillation from Internal Representations
  8 Oct 2019 · 0 repositories · arXiv:1910.03723
• Distilling BERT into Simple Neural Networks with Unlabeled Transfer Data
  4 Oct 2019 · 0 repositories · arXiv:1910.01769
• On the Efficacy of Knowledge Distillation
  3 Oct 2019 · 0 repositories · arXiv:1910.01348
• AntMan: Sparse Low-Rank Compression to Accelerate RNN inference
  2 Oct 2019 · 0 repositories · arXiv:1910.01740
• DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
  2 Oct 2019 · 37 repositories · arXiv:1910.01108
  Syntology official (archive's flag): 1 ran · 21 ran (of which 5 constructed an object rather than computing a result; 13 with no instrument failure: 3 honoured, 1 violated, 9 with no contract checked; 8 where Syntology's instrument failed) · 6 unverified (of 27 harvested samples) · 2 pointer-only (licence)
• Improving Word Embedding Factorization for Compression Using Distilled Nonlinear Neural Decomposition
  2 Oct 2019 · 0 repositories · arXiv:1910.06720
• A Bayesian Optimization Framework for Neural Network Compression
  1 Oct 2019 · 0 repositories
• Distilled Split Deep Neural Networks for Edge-Assisted Real-Time Systems
  1 Oct 2019 · 2 repositories
• Training convolutional neural networks with cheap convolutions and online distillation
  28 Sep 2019 · 1 repository · arXiv:1909.13063
• Compact Trilinear Interaction for Visual Question Answering
  26 Sep 2019 · 1 repository · arXiv:1909.11874
  Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 1 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 5 harvested samples) · 1 pointer-only (licence)
• Collaborative Inter-agent Knowledge Distillation for Reinforcement Learning
  25 Sep 2019 · 0 repositories
• Distilled embedding: non-linear embedding factorization using knowledge distillation
  25 Sep 2019 · 0 repositories
• Extremely Small BERT Models from Mixed-Vocabulary Training
  25 Sep 2019 · 0 repositories · arXiv:1909.11687
• Proactive Sequence Generator via Knowledge Acquisition
  25 Sep 2019 · 0 repositories
• Revisiting Knowledge Distillation via Label Smoothing Regularization
  25 Sep 2019 · 2 repositories · arXiv:1909.11723
  Syntology official (archive's flag): 11 ran · 11 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 0 honoured, 0 violated, 11 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 13 harvested samples)
• SELF-KNOWLEDGE DISTILLATION ADVERSARIAL ATTACK
  25 Sep 2019 · 0 repositories
• XD: Cross-lingual Knowledge Distillation for Polyglot Sentence Embeddings
  25 Sep 2019 · 0 repositories
• FEED: Feature-level Ensemble for Knowledge Distillation
  24 Sep 2019 · 0 repositories · arXiv:1909.10754
• Technical report on Conversational Question Answering
  24 Sep 2019 · 0 repositories · arXiv:1909.10772
• TinyBERT: Distilling BERT for Natural Language Understanding
  23 Sep 2019 · 10 repositories · arXiv:1909.10351
  Syntology official: harvested, nothing ran · 0 ran · 4 unverified (of 4 harvested samples) · 4 pointer-only (licence)
• Positive-Unlabeled Compression on the Cloud
  21 Sep 2019 · 2 repositories · arXiv:1909.09757
• Learning Lightweight Pedestrian Detector with Hierarchical Knowledge Distillation
  20 Sep 2019 · 0 repositories · arXiv:1909.09325
• Ensemble Knowledge Distillation for Learning Improved and Efficient Networks
  17 Sep 2019 · 2 repositories · arXiv:1909.08097
  Syntology 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples)
• Knowledge Distillation for End-to-End Person Search
  3 Sep 2019 · 1 repository · arXiv:1909.01058
• Online Sensor Hallucination via Knowledge Distillation for Multimodal Image Classification
  28 Aug 2019 · 0 repositories · arXiv:1908.10559
• Patient Knowledge Distillation for BERT Model Compression
  25 Aug 2019 · 5 repositories · arXiv:1908.09355
  Syntology community repositories only · 19 ran (of which 0 constructed an object rather than computing a result; 13 with no instrument failure: 2 honoured, 1 violated, 10 with no contract checked; 6 where Syntology's instrument failed) · 8 unverified (of 27 harvested samples) · 27 pointer-only (licence)
• Improved Techniques for Training Adaptive Deep Networks
  17 Aug 2019 · 2 repositories · arXiv:1908.06294
• Knowledge distillation for semi-supervised domain adaptation
  16 Aug 2019 · 0 repositories · arXiv:1908.07355
• Scalable Attentive Sentence-Pair Modeling via Distilled Sentence Embedding
  14 Aug 2019 · 1 repository · arXiv:1908.05161
• Effective Training of Convolutional Neural Networks with Low-bitwidth Weights and Activations
  10 Aug 2019 · 0 repositories · arXiv:1908.04680
• Knowledge Consistency between Neural Networks and Beyond
  5 Aug 2019 · 0 repositories · arXiv:1908.01581
• Learning Lightweight Lane Detection CNNs by Self Attention Distillation
  2 Aug 2019 · 2 repositories · arXiv:1908.00821
  Syntology official (archive's flag): 1 ran · 7 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 8 harvested samples) · 1 pointer-only (licence)
• Self-Knowledge Distillation in Natural Language Processing
  2 Aug 2019 · 0 repositories · arXiv:1908.01851
• GTCOM Neural Machine Translation Systems for WMT19
  1 Aug 2019 · 0 repositories
• PANLP at MEDIQA 2019: Pre-trained Language Models, Transfer Learning and Knowledge Distillation
  1 Aug 2019 · 0 repositories
• The NiuTrans Machine Translation Systems for WMT19
  1 Aug 2019 · 0 repositories
• Distill-to-Label: Weakly Supervised Instance Labeling Using Knowledge Distillation
  26 Jul 2019 · 0 repositories · arXiv:1907.12926
• Distilled Siamese Networks for Visual Tracking
  24 Jul 2019 · 0 repositories · arXiv:1907.10586
• Highlight Every Step: Knowledge Distillation via Collaborative Teaching
  23 Jul 2019 · 1 repository · arXiv:1907.09643
• Lifelong GAN: Continual Learning for Conditional Image Generation
  23 Jul 2019 · 0 repositories · arXiv:1907.10107
• Real-Time Correlation Tracking via Joint Model Compression and Transfer
  23 Jul 2019 · 1 repository · arXiv:1907.09831
• Similarity-Preserving Knowledge Distillation
  23 Jul 2019 · 1 repository · arXiv:1907.09682
  Syntology 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
• Light Multi-segment Activation for Model Compression
  16 Jul 2019 · 2 repositories · arXiv:1907.06870
• Learn Spelling from Teachers: Transferring Knowledge from Language Models to Sequence-to-Sequence Speech Recognition
  13 Jul 2019 · 0 repositories · arXiv:1907.06017
• BAM! Born-Again Multi-Task Networks for Natural Language Understanding
  10 Jul 2019 · 1 repository · arXiv:1907.04829
• Compression of Acoustic Event Detection Models With Quantized Distillation
  1 Jul 2019 · 0 repositories · arXiv:1907.00873
• Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems
  21 Jun 2019 · 2 repositories · arXiv:1906.09308
• GAN-Knowledge Distillation for one-stage Object Detection
  20 Jun 2019 · 0 repositories · arXiv:1906.08467
• Membership Privacy for Machine Learning Models Through Knowledge Transfer
  15 Jun 2019 · 0 repositories · arXiv:1906.06589
• Divide and Conquer: Leveraging Intermediate Feature Representations for Quantized Training of Neural Networks
  14 Jun 2019 · 0 repositories · arXiv:1906.06033
• Scalable Syntax-Aware Language Models Using Knowledge Distillation
  14 Jun 2019 · 0 repositories · arXiv:1906.06438
• Efficient Evaluation-Time Uncertainty Estimation by Improved Distillation
  12 Jun 2019 · 0 repositories · arXiv:1906.05419
• Distilling Object Detectors with Fine-grained Feature Imitation
  9 Jun 2019 · 3 repositories · arXiv:1906.03609
  Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 8 harvested samples)
• When Does Label Smoothing Help?
  6 Jun 2019 · 4 repositories · arXiv:1906.02629
  Syntology 6 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples) · 2 pointer-only (licence)
• Private Deep Learning with Teacher Ensembles
  5 Jun 2019 · 0 repositories · arXiv:1906.02303
• Deep Face Recognition Model Compression via Knowledge Transfer and Distillation
  3 Jun 2019 · 0 repositories · arXiv:1906.00619
• An Adaptive Random Path Selection Approach for Incremental Learning
  3 Jun 2019 · 1 repository · arXiv:1906.01120
• Knowledge Distillation via Instance Relationship Graph
  1 Jun 2019 · 1 repository
• On Knowledge distillation from complex networks for response prediction
  1 Jun 2019 · 0 repositories
• Online Distilling from Checkpoints for Neural Machine Translation
  1 Jun 2019 · 0 repositories
• Structured Knowledge Distillation for Semantic Segmentation
  1 Jun 2019 · 1 repository
• SCAN: A Scalable Neural Networks Framework Towards Compact and Efficient Models
  27 May 2019 · 1 repository · arXiv:1906.03951
• Cross-Resolution Face Recognition via Prior-Aided Face Hallucination and Residual Knowledge Distillation
  26 May 2019 · 0 repositories · arXiv:1905.10777
• Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation
  17 May 2019 · 1 repository · arXiv:1905.08094
• Creating Lightweight Object Detectors with Model Compression for Deployment on Edge Devices
  6 May 2019 · 0 repositories · arXiv:1905.01787