Methods › General › Pruning › Movement Pruning

Movement Pruning

5 papers tagged archive 2025-07-28

Introduced by Victor Sanh et al. in Movement Pruning: Adaptive Sparsity by Fine-Tuning

archive 2025-07-28 Description, source and code snippet are the archive's method entry.

Movement Pruning is a simple, deterministic first-order weight pruning method that is more adaptive to pretrained model fine-tuning. Magnitude pruning can be seen as utilizing zeroth-order information (absolute value) of the running model. In contrast, movement pruning methods are where importance is derived from first-order information. Intuitively, instead of selecting weights that are far from zero, we retain connections that are moving away from zero during the training process.

Paper Source

Papers archive 2025-07-28

5 shown of 5, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.

On Importance of Pruning and Distillation for Efficient Low Resource NLP
21 Sep 2024 · 0 repositories · arXiv:2409.14162
LoRAPrune: Structured Pruning Meets Low-Rank Parameter-Efficient Fine-Tuning
28 May 2023 · 1 repository · arXiv:2305.18403

Syntology ran 0 of 5 samples · 5 unverified
What Matters In The Structured Pruning of Generative Language Models?
7 Feb 2023 · 1 repository · arXiv:2302.03773

Syntology ran 0 of 8 samples · 8 unverified
Block Pruning For Faster Transformers
10 Sep 2021 · 1 repository · arXiv:2109.04838
Movement Pruning: Adaptive Sparsity by Fine-Tuning
15 May 2020 · 4 repositories · arXiv:2005.07683

Syntology ran 4 of 4 samples · 0 unverified · 4 pointer-only (licence)

Tasks archive 2025-07-28

12 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.

Task	Papers
Network Pruning	2
Document Classification	1
GPU	1
Knowledge Distillation	1
Machine Translation	1
Model Compression	1
Question Answering	1
Text Classification	1
Text Generation	1
Transfer Learning	1
parameter-efficient fine-tuning	1
text-classification	1

Usage over time archive 2025-07-28

Papers per year the archive tags with this method, by the paper's archive date (5 dated). Bars are counts, not a trend claim.

Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).

Categories archive 2025-07-28

Pruning

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections