Sparse Learning

Benchmarks archive 2025-07-28

3 leaderboard tables shown for this task, 3 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

3 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 52 papers with code (185 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Variational Dropout Sparsifies Deep Neural Networks
  19 Jan 2017 15 repositories listed Syntology ran 1 of 3 samples · 2 unverified
  We explore a recently proposed Variational Dropout technique that provided an elegant Bayesian interpretation to Gaussian Dropout.
• Rigging the Lottery: Making All Tickets Winners
  25 Nov 2019 11 repositories listed Syntology ran 1 of 1 samples · 0 unverified
  There is a large body of work on training dense networks to yield sparse networks for inference, but this limits the size of the largest trainable sparse model to that of the largest trainable dense model.
• The State of Sparsity in Deep Neural Networks
  25 Feb 2019 6 repositories listed
  We rigorously evaluate three state-of-the-art techniques for inducing sparsity in deep neural networks on two large-scale learning tasks: Transformer trained on WMT 2014 English-to-German, and ResNet-50 trained on…
• Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training
  4 Feb 2021 4 repositories listed
  By starting from a random sparse network and continuously exploring sparse connectivities during training, we can perform an Over-Parameterization in the space-time manifold, closing the gap in the expressibility…
• A General Iterative Shrinkage and Thresholding Algorithm for Non-convex Regularized Optimization Problems
  18 Mar 2013 4 repositories listed Syntology ran 1 of 12 samples · 11 unverified
  A commonly used approach is the Multi-Stage (MS) convex relaxation (or DC programming), which relaxes the original non-convex problem to a sequence of convex problems.
• Controlled Sparsity via Constrained Optimization or: How I Learned to Stop Tuning Penalties and Love Constraints
  8 Aug 2022 2 repositories listed Syntology ran 1 of 1 samples · 0 unverified
  The performance of trained neural networks is robust to harsh levels of pruning.
• abess: A Fast Best Subset Selection Library in Python and R
  19 Oct 2021 2 repositories listed
  In addition, a user-friendly R library is available at the Comprehensive R Archive Network.
• Sparse Training via Boosting Pruning Plasticity with Neuroregeneration
  19 Jun 2021 2 repositories listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  Works on lottery ticket hypothesis (LTH) and single-shot network pruning (SNIP) have raised a lot of attention currently on post-training pruning (iterative magnitude pruning), and before-training pruning (pruning at…
• Sparse Regression at Scale: Branch-and-Bound rooted in First-Order Optimization
  13 Apr 2020 2 repositories listed Syntology ran 0 of 18 samples · 18 unverified
  In this work, we present a new exact MIP framework for ℓ₀ℓ₂-regularized regression that can scale to p ∼10⁷, achieving speedups of at least $5000$x, compared to state-of-the-art exact methods.
• Sparse Networks from Scratch: Faster Training without Losing Performance
  10 Jul 2019 2 repositories listed Syntology ran 0 of 1 samples · 1 unverified
  We demonstrate the possibility of what we call sparse learning: accelerated training of deep neural networks that maintain sparse weights throughout training while achieving dense performance levels.
• Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Science
  15 Jul 2017 2 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 3 pointer-only (licence)
  Through the success of deep learning in various domains, artificial neural networks are currently among the most used artificial intelligence methods.
• Feature Selection: A Data Perspective
  29 Jan 2016 2 repositories listed
  To facilitate and promote the research in this community, we also present an open-source feature selection repository that consists of most of the popular feature selection algorithms (\url{http://featureselection.
• Scalable Subset Selection in Linear Mixed Models
  25 Jun 2025 1 repository listed
  This paper closes the gap with a new ℓ₀ regularized method for LMM subset selection that can run on datasets containing thousands of predictors in seconds to minutes.
• MyESL: Sparse learning in molecular evolution and phylogenetic analysis
  9 Jan 2025 1 repository listed
  The computational core of MyESL is written in C++, which offers model building with or without group sparsity, while the pre- and post-processing of inputs and model outputs is performed using customized functions…
• Optimizing Rare Word Accuracy in Direct Speech Translation with a Retrieval-and-Demonstration Approach
  13 Sep 2024 1 repository listed
  To leverage these valuable resources, we propose a retrieval-and-demonstration approach to enhance rare word translation accuracy in direct ST models.
• Probabilistic Iterative Hard Thresholding for Sparse Learning
  2 Sep 2024 1 repository listed
  For statistical modeling wherein the data regime is unfavorable in terms of dimensionality relative to the sample size, finding hidden sparsity in the ground truth can be critical in formulating an accurate statistical…
• ssProp: Energy-Efficient Training for Convolutional Neural Networks with Scheduled Sparse Back Propagation
  22 Aug 2024 1 repository listed
  Back-propagation (BP) is a major source of computational expense during training deep learning models.
• DRFormer: Multi-Scale Transformer Utilizing Diverse Receptive Fields for Long Time-Series Forecasting
  5 Aug 2024 1 repository listed
  Our proposed model, named DRFormer, is evaluated on various real-world datasets, and experimental results demonstrate its superiority compared to existing methods.
• SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretraining
  4 Jun 2024 1 repository listed Syntology ran 5 of 6 samples · 1 unverified
  In this work, we propose to parameterize the weights as a sum of low-rank and sparse matrices for pretraining, which we call SLTrain.
• CLASSP: a Biologically-Inspired Approach to Continual Learning through Adjustment Suppression and Sparsity Promotion
  29 Apr 2024 1 repository listed
  However, this principle results in a diffuse distribution of updates throughout the model, as it promotes updates for weights that haven't been previously updated, while a sparse update distribution is preferred to…
• Block Sparse Bayesian Learning: A Diversified Scheme
  7 Feb 2024 1 repository listed
  This paper introduces a novel prior called Diversified Block Sparse Prior to characterize the widespread block sparsity phenomenon in real-world data.
• Building explainable graph neural network by sparse learning for the drug-protein binding prediction
  27 Aug 2023 1 repository listed
  Due to the use of the chemical-substructure-based graph, it is guaranteed that any subgraphs in a drug identified by our SLGNN are chemically valid structures.
• HyperSparse Neural Networks: Shifting Exploration to Exploitation through Adaptive Regularization
  14 Aug 2023 1 repository listed Syntology ran 9 of 13 samples · 4 unverified
  Sparse neural networks are a key factor in developing resource-efficient machine learning applications.
• Resource Constrained Model Compression via Minimax Optimization for Spiking Neural Networks
  9 Aug 2023 1 repository listed
  We propose an improved end-to-end Minimax optimization method for this sparse learning problem to better balance the model performance and the computation efficiency.
• Learning to Super-Resolve Blurry Images with Events
  27 Feb 2023 1 repository listed
  Super-Resolution from a single motion Blurred image (SRB) is a severely ill-posed problem due to the joint degradation of motion blurs and low spatial resolution.
• Video-Text Retrieval by Supervised Sparse Multi-Grained Learning
  19 Feb 2023 1 repository listed
  While recent progress in video-text retrieval has been advanced by the exploration of better representation learning, in this paper, we present a novel multi-grained sparse learning framework, S3MA, to learn an aligned…
• Lottery Aware Sparsity Hunting: Enabling Federated Learning on Resource-Limited Edge
  27 Aug 2022 1 repository listed
  A possible solution to this problem is to utilize off-the-shelf sparse learning algorithms at the clients to meet their resource budget.
• AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning
  24 May 2022 1 repository listed
  Standard fine-tuning of large pre-trained language models (PLMs) for downstream tasks requires updating hundreds of millions to billions of parameters, and storing a large copy of the PLM weights for every task…
• APP: Anytime Progressive Pruning
  4 Apr 2022 1 repository listed
  With the latest advances in deep learning, there has been a lot of focus on the online learning paradigm due to its relevance in practical settings.
• L0Learn: A Scalable Package for Sparse Learning using L0 Regularization
  10 Feb 2022 1 repository listed
  We present L0Learn: an open-source package for sparse linear regression and classification using ℓ₀ regularization.

Syntology lines on 10 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.