Methods › Natural Language Processing › Autoencoding Transformers › DeeBERT
DeeBERT
Introduced by Ji Xin et al. in DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
DeeBERT is a method for accelerating BERT inference. It inserts extra classification layers (which are referred to as off-ramps) between each transformer layer of BERT. All transformer layers and off-ramps are jointly fine-tuned on a given downstream dataset. At inference time, after a sample goes through a transformer layer, it is passed to the following off-ramp. If the off-ramp is confident of the prediction, the result is returned; otherwise, the sample is sent to the next transformer layer.
Papers archive 2025-07-28
3 shown of 3, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
SplitEE: Early Exit in Deep Neural Networks with Split Computing 17 Sep 2023 · 1 repository · arXiv:2309.09195
-
RomeBERT: Robust Training of Multi-Exit BERT 24 Jan 2021 · 1 repository · arXiv:2101.09755
-
DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference 27 Apr 2020 · 3 repositories · arXiv:2004.12993Syntology ran 5 of 6 samples · 1 unverified · 5 pointer-only (licence)
Tasks archive 2025-07-28
3 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections