Browse State-of-the-Art › L2 Regularization
L2 Regularization
32 papers with code · 0 benchmarks · 0 datasets archive 2025-07-28
See Weight Decay.
L₂ Regularization or Weight Decay, is a regularization technique applied to the weights of a neural network. We minimize a loss function compromising both the primary loss function and a penalty on the L₂ Norm of the weights:
L_(new)(w) = L_(original)(w) + λwᵀw
where λ is a value determining the strength of the penalty (encouraging smaller weights).
Weight decay can be incorporated directly into the weight update rule, rather than just implicitly by defining it through to objective function. Often weight decay refers to the implementation where we specify it directly in the weight update rule (whereas L2 regularization is usually the implementation which is specified in the objective function).
Description from the archive archive 2025-07-28; Papers-with-Code links inside it are rewritten to this site.
Benchmarks archive 2025-07-28
No benchmark for this task in the archive.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
No dataset record in the archive lists this task.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Papers archive 2025-07-28
archive papers tagged: 128 · with a code link: 32 · where Syntology ran a sample: 9 (5 with a run with no instrument failure, 4 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (9 of 128 tagged: 5 with a run with no instrument failure, 4 where every run was a failure of Syntology's instrument)
Page 1 of 2: papers 1 to 100 of 128, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
30 Oct 2018 3 repositories listed Syntology official (archive's flag): 1 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)Continual learning has received a great deal of attention recently with several approaches being proposed.
-
14 Nov 2023 2 repositories listed Syntology community repositories only · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 2 harvested samples) · 2 pointer-only (licence)The transient nature of ICL is observed in transformers across a range of model sizes and datasets, raising the question of how much to "overtrain" transformers when seeking compact, cheaper-to-run models.
-
26 May 2023 2 repositories listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples)This study investigates how weight decay affects the update behavior of individual neurons in deep neural networks through a combination of applied analysis and experimentation.
-
5 Jun 2025 1 repository listed2D convolutional neural networks (CNNs) have attracted significant attention for hyperspectral image super-resolution tasks.
-
1 Oct 2024 1 repository listedThe WALINET (WAter and LIpid neural NETwork) was compared to conventional methods such as the state-of-the-art lipid L2 regularization and Hankel-Lanczos singular value decomposition (HLSVD) water suppression.
-
15 Mar 2024 1 repository listedIn the wake of the global spread of monkeypox, accurate disease recognition has become crucial.
-
9 Feb 2024 1 repository listedAlthough Multi Armed Bandit (MAB) on one hand and the policy gradient approach on the other hand are among the most used frameworks of Reinforcement Learning, the theoretical properties of the policy gradient algorithm…
-
28 Jan 2024 1 repository listedLogistic regression is a ubiquitous method for probabilistic classification.
-
23 Nov 2023 1 repository listedCommon regularization algorithms for linear regression, such as LASSO and Ridge regression, rely on a regularization hyperparameter that balances the tradeoff between minimizing the fitting error and the norm of the…
-
23 Aug 2023 1 repository listedIn this work, we introduce channel-wise l1/l2 group sparsity in the shared convolutional layers parameters (or weights) of the multi-task learning model.
-
23 Jun 2023 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 5 harvested samples) · 3 pointer-only (licence)If deep-learning systems are applied in a continual learning setting, then it is well known that they may fail to remember earlier examples.
-
22 May 2023 1 repository listedInspired by this analysis, we propose simple-yet-effective linear autoencoder models using diagonal inequality constraints, called Relaxed Linear AutoEncoder (RLAE) and Relaxed Denoising Linear AutoEncoder (RDLAE).
-
16 Dec 2022 1 repository listedLanguage models are widely deployed to provide automatic text completion services in user products.
-
Motion Correction and Volumetric Reconstruction for Fetal Functional Magnetic Resonance Imaging Data11 Feb 2022 1 repository listedHere, we propose a novel framework, which estimates a high-resolution reference volume by using outlier-robust motion correction, and by utilizing Huber L2 regularization for intra-stack volumetric reconstruction of the…
-
31 Dec 2021 1 repository listedIn practice, multi-task learning (through learning features shared among tasks) is an essential property of deep neural networks (NNs).
-
11 Oct 2021 1 repository listedThis active regularization makes use of the model behavior during training to regularize it in a more directed manner.
-
15 Sep 2021 1 repository listedWe demonstrate on a simple string editing task and a machine translation task that the Transformer model performance drops significantly when facing sequences of length diverging from the length distribution in the…
-
11 Jun 2021 1 repository listedOur analysis in this paper decouples capacity and width via the generalization of neural networks to Deep Gaussian Processes (Deep GP), a class of nonparametric hierarchical models that subsume neural nets.
-
2 Mar 2021 1 repository listedDue to the over-parameterization nature, neural networks are a powerful tool for nonlinear function approximation.
-
30 Dec 2020 1 repository listedIn this paper, we present an unsupervised image enhancement generative adversarial network (UEGAN), which learns the corresponding image-to-image mapping from a set of images with desired characteristics in an…
-
16 Dec 2020 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified; the one sample that ran constructed an object rather than computing a result (of 3 harvested samples) · 3 pointer-only (licence)Regularization has long been utilized to learn sparsity in deep neural network pruning.
-
28 Jul 2020 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)We empirically show that our label-only membership inference attacks perform on par with prior attacks that required access to model confidences.
-
1 May 2020 1 repository listedDistributionally robust optimization (DRO) allows us to learn models that instead minimize the worst-case training loss over a set of pre-defined groups.
-
17 Feb 2020 1 repository listedOur results reveal the relationship between MIA accuracy and properties of the dataset and training model in use.
-
29 Sep 2019 1 repository listed Syntology 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)There are existing efforts that model the training dynamics of GANs in the parameter space but the analysis cannot directly motivate practically effective stabilizing methods.
-
5 Mar 2019 1 repository listedWe propose a smooth kernel regularizer that encourages spatial correlations in convolution kernel weights.
-
8 Dec 2018 1 repository listedImportance-weighted risk minimization is a key ingredient in many machine learning algorithms for causal inference, domain adaptation, class imbalance, and off-policy reinforcement learning.
-
6 Dec 2018 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 1 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)In this paper, we investigate the problem of overfitting in deep reinforcement learning.
-
21 Feb 2018 1 repository listedWe propose a deep collaborative weight-based classification (DeepCWC) method to resolve this problem, by providing a novel option to fully take advantage of deep features in classic machine learning.
-
22 Apr 2017 1 repository listedWe have developed convolutional neural networks (CNN) for a facial expression recognition task.
-
22 Jan 2017 1 repository listed
-
31 Jul 2016 1 repository listed
-
Overcoming catastrophic forgetting in neural networks14 Jul 2025 0 repositories listed
-
From large-eddy simulations to deep learning: A U-net model for fast urban canopy flow predictions9 Jul 2025 0 repositories listed
-
Geometry of Learning -- L2 Phase Transitions in Deep and Shallow Neural Networks10 May 2025 0 repositories listed
-
Deep Learning in Renewable Energy Forecasting: A Cross-Dataset Evaluation of Temporal and Spatial Models6 May 2025 0 repositories listed
-
Understand the Effect of Importance Weighting in Deep Learning on Dataset Shift6 May 2025 0 repositories listed
-
Semantic segmentation for building houses from wooden cubes28 Mar 2025 0 repositories listed
-
GPT Meets Graphs and KAN Splines: Testing Novel Frameworks on Multitask Fine-Tuned GPT-2 with LoRA25 Mar 2025 0 repositories listed
-
CtrTab: Tabular Data Synthesis with High-Dimensional and Limited Data9 Mar 2025 0 repositories listed
-
Low-rank bias, weight decay, and model merging in neural networks24 Feb 2025 0 repositories listed
-
Multimodal Bearing Fault Classification Under Variable Conditions: A 1D CNN with Transfer Learning23 Feb 2025 0 repositories listed
-
Renewable Energy Prediction: A Comparative Study of Deep Learning Models for Complex Dataset Analysis27 Jan 2025 0 repositories listed
-
Learning in Log-Domain: Subthreshold Analog AI Accelerator Based on Stochastic Gradient Descent22 Jan 2025 0 repositories listed
-
Super-Resolution for Remote Sensing Imagery via the Coupling of a Variational Model and Deep Learning13 Dec 2024 0 repositories listed
-
Parkinson's Disease Diagnosis Through Deep Learning: A Novel LSTM-Based Approach for Freezing of Gait Detection9 Dec 2024 0 repositories listed
-
Effectiveness of L2 Regularization in Privacy-Preserving Machine Learning2 Dec 2024 0 repositories listed
-
Analysis of High-dimensional Gaussian Labeled-unlabeled Mixture Model via Message-passing Algorithm29 Nov 2024 0 repositories listed
-
Recurrent Stochastic Configuration Networks with Hybrid Regularization for Nonlinear Dynamics Modelling26 Nov 2024 0 repositories listed
-
Carbon price fluctuation prediction using blockchain information A new hybrid machine learning approach5 Nov 2024 0 repositories listed
-
Weight decay induces low-rank attention layers31 Oct 2024 0 repositories listed
-
Rethinking Conventional Wisdom in Machine Learning: From Generalization to Scaling23 Sep 2024 0 repositories listed
-
Training Dynamics of Nonlinear Contrastive Learning Model in the High Dimensional Limit11 Jun 2024 0 repositories listed
-
Comparative Study of Bitcoin Price Prediction13 May 2024 0 repositories listed
-
Derivative-based regularization for regression1 May 2024 0 repositories listed
-
An Experiment on Feature Selection using Logistic Regression31 Jan 2024 0 repositories listed
-
Reverse Engineering Deep ReLU Networks An Optimization-based Algorithm7 Dec 2023 0 repositories listed
-
On sparse regression, Lp-regularization, and automated model discovery9 Oct 2023 0 repositories listed
-
Maintaining Plasticity in Continual Learning via Regenerative Regularization23 Aug 2023 0 repositories listed
-
Dropout Regularization Versus ℓ₂-Penalization in the Linear Model18 Jun 2023 0 repositories listed
-
Electromyography Signal Classification Using Deep Learning6 May 2023 0 repositories listed
-
Maximum margin learning of t-SPNs for cell classification with filtered input16 Mar 2023 0 repositories listed
-
Emphasizing Unseen Words: New Vocabulary Acquisition for End-to-End Speech Recognition20 Feb 2023 0 repositories listed
-
Globally Gated Deep Linear Networks31 Oct 2022 0 repositories listed
-
Linking Neural Collapse and L2 Normalization with Improved Out-of-Distribution Detection in Deep Neural Networks17 Sep 2022 0 repositories listed
-
On the utility and protection of optimization with differential privacy and classic regularization techniques7 Sep 2022 0 repositories listed
-
Perturbation of Deep Autoencoder Weights for Model Compression and Classification of Tabular Data17 May 2022 0 repositories listed
-
15 May 2022 0 repositories listed
-
A Note on the Regularity of Images Generated by Convolutional Neural Networks22 Apr 2022 0 repositories listed
-
A Closer Look at Rehearsal-Free Continual Learning31 Mar 2022 0 repositories listed
-
Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learning22 Oct 2021 0 repositories listed
-
Regularized Training of Nearest Neighbor Language Models16 Sep 2021 0 repositories listed
-
Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity30 Jun 2021 0 repositories listed
-
12 Jun 2021 0 repositories listed Syntology 4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 5 harvested samples) · 5 pointer-only (licence)
-
Effect of the regularization hyperparameter on deep learning-based segmentation in LGE-MRI10 Dec 2020 0 repositories listed
-
Gram Regularization for Multi-view 3D Shape Retrieval16 Nov 2020 0 repositories listed
-
Exponentially Weighted l_2 Regularization Strategy in Constructing Reinforced Second-order Fuzzy Rule-based Model2 Jul 2020 0 repositories listed
-
An FPGA-Based On-Device Reinforcement Learning Approach using Online Sequential Learning10 May 2020 0 repositories listed
-
A Bayesian traction force microscopy method with automated denoising in a user-friendly software package4 May 2020 0 repositories listed
-
Data-dependent Gaussian Prior Objective for Language Generation1 May 2020 0 repositories listed
-
Correlated Initialization for Correlated Data9 Mar 2020 0 repositories listed
-
Tighter Bound Estimation of Sensitivity Analysis for Incremental and Decremental Data Modification6 Mar 2020 0 repositories listed
-
Regularisation Can Mitigate Poisoning Attacks: A Novel Analysis Based on Multiobjective Bilevel Optimisation28 Feb 2020 0 repositories listed
-
Empirical Study on Airline Delay Analysis and Prediction17 Feb 2020 0 repositories listed
-
Self-Distillation Amplifies Regularization in Hilbert Space13 Feb 2020 0 repositories listed
-
Customers Churn Prediction in Financial Institution Using Artificial Neural Network23 Dec 2019 0 repositories listed
-
A Comparative Study of Neural Network Compression24 Oct 2019 0 repositories listed
-
Improved error rates for sparse (group) learning with Lipschitz loss functions20 Oct 2019 0 repositories listed
-
The Ant Swarm Neuro-Evolution Procedure for Optimizing Recurrent Networks26 Sep 2019 0 repositories listed
-
Unsupervised Video Depth Estimation Based on Ego-motion and Disparity Consensus3 Sep 2019 0 repositories listed
-
The Theory Behind Overfitting, Cross Validation, Regularization, Bagging, and Boosting: Tutorial28 May 2019 0 repositories listed
-
Emergence of Implicit Filter Sparsity in Convolutional Neural Networks17 May 2019 0 repositories listed
-
Implicit Filter Sparsification In Convolutional Neural Networks13 May 2019 0 repositories listed
-
A MAX-AFFINE SPLINE PERSPECTIVE OF RECURRENT NEURAL NETWORKS1 May 2019 0 repositories listed
-
Analysis of overfitting in the regularized Cox model14 Apr 2019 0 repositories listed
-
Deep Optimization model for Screen Content Image Quality Assessment using Neural Networks2 Mar 2019 0 repositories listed
-
Adaptive Estimators Show Information Compression in Deep Neural Networks24 Feb 2019 0 repositories listed
-
Multi-branch fusion network for hyperspectral image classification18 Jan 2019 0 repositories listed
-
Construction of Differentially Private Empirical Distributions from a low-order Marginals Set through Solving Linear Equations with l2 Regularization12 Dec 2018 0 repositories listed
-
On Implicit Filter Level Sparsity in Convolutional Neural Networks29 Nov 2018 0 repositories listed
Syntology lines on 9 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections