Methods › General › Regularization › Weight Decay
Weight Decay
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Weight Decay, or L₂ Regularization, is a regularization technique applied to the weights of a neural network. We minimize a loss function compromising both the primary loss function and a penalty on the L₂ Norm of the weights:
L_(new)(w) = L_(original)(w) + λwᵀw
where λ is a value determining the strength of the penalty (encouraging smaller weights).
Weight decay can be incorporated directly into the weight update rule, rather than just implicitly by defining it through to objective function. Often weight decay refers to the implementation where we specify it directly in the weight update rule (whereas L2 regularization is usually the implementation which is specified in the objective function).
Image Source: Deep Learning, Goodfellow et al
Papers archive 2025-07-28
30 shown of 10,713, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
Constructing and Evaluating Declarative RAG Pipelines in PyTerrier 12 Jun 2025 · 1 repository · arXiv:2506.10802
-
A Novel Lightweight Transformer with Edge-Aware Fusion for Remote Sensing Image Captioning 11 Jun 2025 · 0 repositories · arXiv:2506.09429
-
Auto-Compressing Networks 11 Jun 2025 · 0 repositories · arXiv:2506.09714
-
Learning Efficient and Generalizable Graph Retriever for Knowledge-Graph Question Answering 11 Jun 2025 · 1 repository · arXiv:2506.09645
-
Sampling Theory for Super-Resolution with Implicit Neural Representations 11 Jun 2025 · 1 repository · arXiv:2506.09949
-
Unsupervised Deep Clustering of MNIST with Triplet-Enhanced Convolutional Autoencoders 11 Jun 2025 · 0 repositories · arXiv:2506.10094
-
LlamaRec-LKG-RAG: A Single-Pass, Learnable Knowledge Graph-RAG Framework for LLM-Based Ranking 9 Jun 2025 · 1 repository · arXiv:2506.07449
-
LLM-driven Indoor Scene Layout Generation via Scaled Human-aligned Data Synthesis and Multi-Stage Preference Optimization 9 Jun 2025 · 0 repositories · arXiv:2506.07570
-
SceneRAG: Scene-level Retrieval-Augmented Generation for Video Understanding 9 Jun 2025 · 0 repositories · arXiv:2506.07600
-
Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models 8 Jun 2025 · 0 repositories · arXiv:2506.07121
-
Direct Behavior Optimization: Unlocking the Potential of Lightweight LLMs 6 Jun 2025 · 0 repositories · arXiv:2506.06401
-
When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation 6 Jun 2025 · 1 repository · arXiv:2506.05690Syntology ran 0 of 5 samples · 5 unverified
-
Grapheme-Coherent Phonemic and Prosodic Annotation of Speech by Implicit and Explicit Grapheme Conditioning 5 Jun 2025 · 0 repositories · arXiv:2506.04527
-
On Automating Security Policies with Contemporary LLMs 5 Jun 2025 · 0 repositories · arXiv:2506.04838
-
The NTNU System at the S&I Challenge 2025 SLA Open Track 5 Jun 2025 · 0 repositories · arXiv:2506.05121
-
Facts are Harder Than Opinions -- A Multilingual, Comparative Analysis of LLM-Based Fact-Checking Reliability 4 Jun 2025 · 0 repositories · arXiv:2506.03655
-
Lions and Muons: Optimization via Stochastic Frank-Wolfe 4 Jun 2025 · 0 repositories · arXiv:2506.04192
-
Magic Mushroom: A Customizable Benchmark for Fine-grained Analysis of Retrieval Noise Erosion in RAG Systems 4 Jun 2025 · 0 repositories · arXiv:2506.03901
-
Privacy and Security Threat for OpenAI GPTs 4 Jun 2025 · 0 repositories · arXiv:2506.04036
-
TracLLM: A Generic Framework for Attributing Long Context LLMs 4 Jun 2025 · 1 repository · arXiv:2506.04202
-
A Novel Deep Reinforcement Learning Method for Computation Offloading in Multi-User Mobile Edge Computing with Decentralization 3 Jun 2025 · 0 repositories · arXiv:2506.02458
-
An Exploratory Framework for Future SETI Applications: Detecting Generative Reactivity via Language Models 3 Jun 2025 · 0 repositories · arXiv:2506.02730
-
Enhancing Automatic PT Tagging for MEDLINE Citations Using Transformer-Based Models 3 Jun 2025 · 0 repositories · arXiv:2506.03321
-
Rethinking the effects of data contamination in Code Intelligence 3 Jun 2025 · 0 repositories · arXiv:2506.02791
-
LLMs as World Models: Data-Driven and Human-Centered Pre-Event Simulation for Disaster Impact Assessment 2 Jun 2025 · 0 repositories · arXiv:2506.06355
-
Retrieval-Augmented Generation of Ontologies from Relational Databases 2 Jun 2025 · 0 repositories · arXiv:2506.01232
-
How Neural Networks Organize Concepts: Introducing Concept Trajectory Analysis for Deep Learning Interpretability 1 Jun 2025 · 1 repository
-
RARE: Retrieval-Aware Robustness Evaluation for Retrieval-Augmented Generation Systems 1 Jun 2025 · 1 repository · arXiv:2506.00789
-
FinBERT2: A Specialized Bidirectional Encoder for Bridging the Gap in Finance-Specific Deployment of Large Language Models 31 May 2025 · 0 repositories · arXiv:2506.06335
-
Adversarial Threat Vectors and Risk Mitigation for Retrieval-Augmented Generation Systems 30 May 2025 · 0 repositories · arXiv:2506.00281
Tasks archive 2025-07-28
20 shown of 1,523 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
| Task | Papers |
|---|---|
| Language Modelling | 1,799 |
| Language Modeling | 1,406 |
| Retrieval | 1,368 |
| RAG | 1,244 |
| Retrieval-augmented Generation | 1,068 |
| Question Answering | 996 |
| Sentence | 866 |
| Large Language Model | 480 |
| Sentiment Analysis | 458 |
| Text Classification | 448 |
| Text Generation | 417 |
| text-classification | 397 |
| Transfer Learning | 369 |
| Classification | 300 |
| Information Retrieval | 291 |
| Natural Language Understanding | 285 |
| Word Embeddings | 274 |
| Named Entity Recognition | 264 |
| Articles | 255 |
| named-entity-recognition | 254 |
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections