</> Code with Papers by Syntology

Methods › General › Knowledge Distillation › Hydra

Hydra

34 papers tagged archive 2025-07-28

archive 2025-07-28 Description, source and code snippet are the archive's method entry.

Hydra is a multi-headed neural network for model distillation with a shared body network. The shared body network learns a joint feature representation that enables each head to capture the predictive behavior of each ensemble member. Existing distillation methods often train a distillation network to imitate the prediction of a larger network. Hydra instead learns to distill the individual predictions of each ensemble member into separate light-weight head models while amortizing the computation through a shared heavy-weight body network. This retains the diversity of ensemble member predictions which is otherwise lost in knowledge distillation.

Source: Hydra: Preserving Ensemble Diversity for Model Distillation

Papers archive 2025-07-28

30 shown of 34, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.

Tasks archive 2025-07-28

20 shown of 51 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.

TaskPapers
Language Modelling4
Recommendation Systems4
model4
Benchmarking3
Language Modeling3
Mamba3
Deep Learning2
Model Selection2
Scheduling2
Sequential Recommendation2
Transfer Learning2
3D Multi-Object Tracking1
3D Object Detection1
3D Object Detection (RoI)1
3D Reconstruction1
3D Semantic Occupancy Prediction1
Adversarial Attack1
Adversarial Defense1
Adversarial Robustness1
Anomaly Detection1

Usage over time archive 2025-07-28

Papers per year tagged with Hydra: 2020 to 2025, peak 15 15 0 2020: 1 paper 2020 2021: 2 papers 2021 2022: 3 papers 2022 2023: 7 papers 2023 2024: 15 papers 2024 2025: 6 papers 2025
Papers per year the archive tags with this method, by the paper's archive date (34 dated). Bars are counts, not a trend claim.

Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).

Categories archive 2025-07-28

Knowledge Distillation

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections