Network Pruning

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

5 leaderboard tables shown for this task, 5 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

5 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 239 papers with code (534 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
  24 Feb 2016 59 repositories listed Syntology ran 4 of 4 samples · 0 unverified · 2 pointer-only (licence)
  (2) Smaller DNNs require less bandwidth to export a new model from the cloud to an autonomous car.
• The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
  9 Mar 2018 24 repositories listed Syntology ran 4 of 25 samples · 21 unverified · 6 pointer-only (licence)
  Based on these results, we articulate the "lottery ticket hypothesis:" dense, randomly-initialized, feed-forward networks contain subnetworks ("winning tickets") that - when trained in isolation - reach test accuracy…
• Pruning Filters for Efficient ConvNets
  31 Aug 2016 21 repositories listed Syntology ran 20 of 36 samples · 16 unverified · 20 pointer-only (licence)
  However, magnitude-based pruning of weights reduces a significant number of parameters from the fully connected layers and may not adequately reduce the computation costs in the convolutional layers due to irregular…
• Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
  1 Oct 2015 15 repositories listed Syntology ran 2 of 4 samples · 2 unverified · 1 pointer-only (licence)
  To address this limitation, we introduce "deep compression", a three stage pipeline: pruning, trained quantization and Huffman coding, that work together to reduce the storage requirement of neural networks by 35x to…
• SNIP: Single-shot Network Pruning based on Connection Sensitivity
  4 Oct 2018 8 repositories listed Syntology ran 5 of 12 samples · 7 unverified · 7 pointer-only (licence)
  To achieve this, we introduce a saliency criterion based on connection sensitivity that identifies structurally important connections in the network for the given task.
• A Simple and Effective Pruning Approach for Large Language Models
  20 Jun 2023 7 repositories listed Syntology ran 9 of 22 samples · 13 unverified · 9 pointer-only (licence)
  Motivated by the recent observation of emergent large magnitude features in LLMs, our approach prunes weights with the smallest magnitudes multiplied by the corresponding input activations, on a per-output basis.
• Manifold Regularized Dynamic Network Pruning
  10 Mar 2021 7 repositories listed Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
  Then, the manifold relationship between instances and the pruned sub-networks will be aligned in the training procedure.
• SCOP: Scientific Control for Reliable Neural Network Pruning
  21 Oct 2020 4 repositories listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  To increase the reliability of the results, we prefer to have a more rigorous research design by including a scientific control group as an essential part to minimize the effect of all factors except the association…
• Movement Pruning: Adaptive Sparsity by Fine-Tuning
  15 May 2020 4 repositories listed Syntology ran 4 of 4 samples · 0 unverified · 4 pointer-only (licence)
  Magnitude pruning is a widely used strategy for reducing model size in pure supervised learning; however, it is less effective in the transfer learning regime that has become standard for state-of-the-art natural…
• Similarity of Neural Networks with Gradients
  25 Mar 2020 4 repositories listed Syntology ran 0 of 11 samples · 11 unverified
  A suitable similarity index for comparing learnt neural networks plays an important role in understanding the behaviour of the highly-nonlinear functions, and can provide insights on further theoretical analysis and…
• HYDRA: Pruning Adversarially Robust Neural Networks
  24 Feb 2020 4 repositories listed Syntology ran 4 of 19 samples · 15 unverified · 17 pointer-only (licence)
  We demonstrate that our approach, titled HYDRA, achieves compressed networks with state-of-the-art benign and robust accuracy, simultaneously.
• Discovering Neural Wirings
  3 Jun 2019 4 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 3 pointer-only (licence)
  In this work we propose a method for discovering neural wirings.
• Network Pruning via Transformable Architecture Search
  23 May 2019 4 repositories listed
  The maximum probability for the size in each distribution serves as the width and depth of the pruned network, whose parameters are learned by knowledge transfer, e.
• A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers
  10 Apr 2018 4 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 3 pointer-only (licence)
  We first formulate the weight pruning problem of DNNs as a nonconvex optimization problem with combinatorial constraints specifying the sparsity requirements, and then adopt the ADMM framework for systematic weight…
• PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning
  15 Nov 2017 4 repositories listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  This paper presents a method for adding multiple tasks to a single deep neural network while avoiding catastrophic forgetting.
• Picking Winning Tickets Before Training by Preserving Gradient Flow
  18 Feb 2020 3 repositories listed Syntology ran 1 of 2 samples · 1 unverified
  Overparameterization has been shown to benefit both the optimization and generalization of neural networks, but large networks are resource hungry at both training and test time.
• Importance Estimation for Neural Network Pruning
  25 Jun 2019 3 repositories listed
  On ResNet-101, we achieve a 40% FLOPS reduction by removing 30% of the parameters, with a loss of 0.
• Adversarial Fine-tuning of Compressed Neural Networks for Joint Improvement of Robustness and Efficiency
  14 Mar 2024 2 repositories listed
  We present experiments on two benchmark datasets showing that adversarial fine-tuning of compressed models can achieve robustness performance comparable to adversarially trained models, while also improving…
• FALCON: FLOP-Aware Combinatorial Optimization for Neural Network Pruning
  11 Mar 2024 2 repositories listed
  In this paper, we propose FALCON, a novel combinatorial-optimization-based framework for network pruning that jointly takes into account model accuracy (fidelity), FLOPs, and sparsity constraints.
• Do Localization Methods Actually Localize Memorized Data in LLMs? A Tale of Two Benchmarks
  15 Nov 2023 2 repositories listed Syntology ran 6 of 15 samples · 9 unverified
  On the other hand, even successful methods identify neurons that are not specific to a single memorized sequence.
• Beyond Size: How Gradients Shape Pruning Decisions in Large Language Models
  8 Nov 2023 2 repositories listed Syntology ran 3 of 7 samples · 4 unverified
  GBLM-Pruner leverages the first-order term of the Taylor expansion, operating in a training-free manner by harnessing properly normalized gradients from a few calibration samples to determine the pruning metric, and…
• Improving the Transferability of Adversarial Examples via Direction Tuning
  27 Mar 2023 2 repositories listed
  Although considerable efforts have been developed on improving the transferability of adversarial examples generated by transfer-based adversarial attacks, our investigation found that, the big deviation between the…
• Iterative Soft Shrinkage Learning for Efficient Image Super-Resolution
  16 Mar 2023 2 repositories listed
  Second, existing pruning methods generally operate upon a pre-trained network for the sparse structure determination, hard to get rid of dense model training in the traditional SR paradigm.
• UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers
  31 Jan 2023 2 repositories listed Syntology ran 0 of 2 samples · 2 unverified
  Real-world data contains a vast amount of multimodal information, among which vision and language are the two most representative modalities.
• Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning
  12 Jan 2023 2 repositories listed
  The state of neural network pruning has been noticed to be unclear and even confusing for a while, largely due to "a lack of standardized benchmarks and metrics" [3].
• Training Deep Neural Networks with Joint Quantization and Pruning of Weights and Activations
  15 Oct 2021 2 repositories listed
  State-of-the-art quantization techniques are currently applied to both the weights and activations; however, pruning is most often applied to only the weights of the network.
• Group Fisher Pruning for Practical Network Compression
  2 Aug 2021 2 repositories listed
  Our method can be used to prune any structures including those with coupled channels.
• Manipulating Identical Filter Redundancy for Efficient Pruning on Deep and Complicated CNN
  30 Jul 2021 2 repositories listed
  The existence of redundancy in Convolutional Neural Networks (CNNs) enables us to remove some filters/channels with acceptable performance drops.
• M-FAC: Efficient Matrix-Free Approximations of Second-Order Information
  7 Jul 2021 2 repositories listed Syntology ran 7 of 14 samples · 7 unverified
  We propose two new algorithms as part of a framework called M-FAC: the first algorithm is tailored towards network compression and can compute the IHVP for dimension d, if the Hessian is given as a sum of m rank-one…
• AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks
  23 Jun 2021 2 repositories listed Syntology ran 2 of 5 samples · 3 unverified
  The increasing computational requirements of deep neural networks (DNNs) have led to significant interest in obtaining DNN models that are sparse, yet accurate.

Syntology lines on 20 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.