Methods › General › Regularization › Weight Decay
Weight Decay
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Weight Decay, or L₂ Regularization, is a regularization technique applied to the weights of a neural network. We minimize a loss function compromising both the primary loss function and a penalty on the L₂ Norm of the weights:
L_(new)(w) = L_(original)(w) + λwᵀw
where λ is a value determining the strength of the penalty (encouraging smaller weights).
Weight decay can be incorporated directly into the weight update rule, rather than just implicitly by defining it through to objective function. Often weight decay refers to the implementation where we specify it directly in the weight update rule (whereas L2 regularization is usually the implementation which is specified in the objective function).
Image Source: Deep Learning, Goodfellow et al
In Syntology name not resolved (unknown_name). The lookup matched no method, paper or venue by this name; that asserts nothing about whether code exists.
Near matches the lookup lists, each a different name: Sample Weight Decay (SWD); Anchored Weight Decay (AWD); Adaptive Decay Weight (ADW); Scheduled Weight Decay (SWD); Weighted Temporal Decay Loss; Positional Decay Reweighting (PDR); Decoupled Weight Decay Regularization; Decay RNN.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 10,713 · with a code link: 4,533 · where Syntology ran a sample: 1,291 (1,031 with a run with no instrument failure, 260 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (1,291 of 10,713 tagged: 1,031 with a run with no instrument failure, 260 where every run was a failure of Syntology's instrument)
Page 1 of 108: papers 1 to 100 of 10,713, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
Constructing and Evaluating Declarative RAG Pipelines in PyTerrier 12 Jun 2025 · 1 repository · arXiv:2506.10802
-
A Novel Lightweight Transformer with Edge-Aware Fusion for Remote Sensing Image Captioning 11 Jun 2025 · 0 repositories · arXiv:2506.09429
-
Auto-Compressing Networks 11 Jun 2025 · 0 repositories · arXiv:2506.09714
-
Learning Efficient and Generalizable Graph Retriever for Knowledge-Graph Question Answering 11 Jun 2025 · 1 repository · arXiv:2506.09645
-
Sampling Theory for Super-Resolution with Implicit Neural Representations 11 Jun 2025 · 1 repository · arXiv:2506.09949
-
Unsupervised Deep Clustering of MNIST with Triplet-Enhanced Convolutional Autoencoders 11 Jun 2025 · 0 repositories · arXiv:2506.10094
-
LlamaRec-LKG-RAG: A Single-Pass, Learnable Knowledge Graph-RAG Framework for LLM-Based Ranking 9 Jun 2025 · 1 repository · arXiv:2506.07449
-
LLM-driven Indoor Scene Layout Generation via Scaled Human-aligned Data Synthesis and Multi-Stage Preference Optimization 9 Jun 2025 · 0 repositories · arXiv:2506.07570
-
SceneRAG: Scene-level Retrieval-Augmented Generation for Video Understanding 9 Jun 2025 · 0 repositories · arXiv:2506.07600
-
Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models 8 Jun 2025 · 0 repositories · arXiv:2506.07121
-
Direct Behavior Optimization: Unlocking the Potential of Lightweight LLMs 6 Jun 2025 · 0 repositories · arXiv:2506.06401
-
When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation 6 Jun 2025 · 1 repository · arXiv:2506.05690Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples)
-
Grapheme-Coherent Phonemic and Prosodic Annotation of Speech by Implicit and Explicit Grapheme Conditioning 5 Jun 2025 · 0 repositories · arXiv:2506.04527
-
On Automating Security Policies with Contemporary LLMs 5 Jun 2025 · 0 repositories · arXiv:2506.04838
-
The NTNU System at the S&I Challenge 2025 SLA Open Track 5 Jun 2025 · 0 repositories · arXiv:2506.05121
-
Facts are Harder Than Opinions -- A Multilingual, Comparative Analysis of LLM-Based Fact-Checking Reliability 4 Jun 2025 · 0 repositories · arXiv:2506.03655
-
Lions and Muons: Optimization via Stochastic Frank-Wolfe 4 Jun 2025 · 0 repositories · arXiv:2506.04192
-
Magic Mushroom: A Customizable Benchmark for Fine-grained Analysis of Retrieval Noise Erosion in RAG Systems 4 Jun 2025 · 0 repositories · arXiv:2506.03901
-
Privacy and Security Threat for OpenAI GPTs 4 Jun 2025 · 0 repositories · arXiv:2506.04036
-
TracLLM: A Generic Framework for Attributing Long Context LLMs 4 Jun 2025 · 1 repository · arXiv:2506.04202
-
A Novel Deep Reinforcement Learning Method for Computation Offloading in Multi-User Mobile Edge Computing with Decentralization 3 Jun 2025 · 0 repositories · arXiv:2506.02458
-
An Exploratory Framework for Future SETI Applications: Detecting Generative Reactivity via Language Models 3 Jun 2025 · 0 repositories · arXiv:2506.02730
-
Enhancing Automatic PT Tagging for MEDLINE Citations Using Transformer-Based Models 3 Jun 2025 · 0 repositories · arXiv:2506.03321
-
Rethinking the effects of data contamination in Code Intelligence 3 Jun 2025 · 0 repositories · arXiv:2506.02791
-
LLMs as World Models: Data-Driven and Human-Centered Pre-Event Simulation for Disaster Impact Assessment 2 Jun 2025 · 0 repositories · arXiv:2506.06355
-
Retrieval-Augmented Generation of Ontologies from Relational Databases 2 Jun 2025 · 0 repositories · arXiv:2506.01232
-
How Neural Networks Organize Concepts: Introducing Concept Trajectory Analysis for Deep Learning Interpretability 1 Jun 2025 · 1 repository
-
RARE: Retrieval-Aware Robustness Evaluation for Retrieval-Augmented Generation Systems 1 Jun 2025 · 1 repository · arXiv:2506.00789
-
FinBERT2: A Specialized Bidirectional Encoder for Bridging the Gap in Finance-Specific Deployment of Large Language Models 31 May 2025 · 0 repositories · arXiv:2506.06335
-
Adversarial Threat Vectors and Risk Mitigation for Retrieval-Augmented Generation Systems 30 May 2025 · 0 repositories · arXiv:2506.00281
-
Agent-X: Evaluating Deep Multimodal Reasoning in Vision-Centric Agentic Tasks 30 May 2025 · 1 repository · arXiv:2505.24876
-
ClueAnchor: Clue-Anchored Knowledge Reasoning Exploration and Optimization for Retrieval-Augmented Generation 30 May 2025 · 1 repository · arXiv:2505.24388
-
Interpretable phenotyping of Heart Failure patients with Dutch discharge letters 30 May 2025 · 0 repositories · arXiv:2505.24619
-
LPASS: Linear Probes as Stepping Stones for vulnerability detection using compressed LLMs 30 May 2025 · 0 repositories · arXiv:2505.24451
-
MOFGPT: Generative Design of Metal-Organic Frameworks using Language Models 30 May 2025 · 1 repository · arXiv:2506.00198
-
RealDrive: Retrieval-Augmented Driving with Diffusion Models 30 May 2025 · 0 repositories · arXiv:2505.24808
-
When GPT Spills the Tea: Comprehensive Assessment of Knowledge File Leakage in GPTs 30 May 2025 · 0 repositories · arXiv:2506.00197
-
Bridging the Gap Between Semantic and User Preference Spaces for Multi-modal Music Representation Learning 29 May 2025 · 0 repositories · arXiv:2505.23298
-
Critical Batch Size Revisited: A Simple Empirical Approach to Large-Batch Language Model Training 29 May 2025 · 0 repositories · arXiv:2505.23971
-
Data-efficient Meta-models for Evaluation of Context-based Questions and Answers in LLMs 29 May 2025 · 0 repositories · arXiv:2505.23299
-
Daunce: Data Attribution through Uncertainty Estimation 29 May 2025 · 0 repositories · arXiv:2505.23223
-
Enhancing LLM-Based Code Generation with Complexity Metrics: A Feedback-Driven Approach 29 May 2025 · 0 repositories · arXiv:2505.23953
-
Evaluating AI capabilities in detecting conspiracy theories on YouTube 29 May 2025 · 1 repository · arXiv:2505.23570
-
Matryoshka Model Learning for Improved Elastic Student Models 29 May 2025 · 0 repositories · arXiv:2505.23337
-
MCP Safety Training: Learning to Refuse Falsely Benign MCP Exploits using Improved Preference Alignment 29 May 2025 · 0 repositories · arXiv:2505.23634
-
Query Routing for Retrieval-Augmented Language Models 29 May 2025 · 0 repositories · arXiv:2505.23052
-
Reducing Latency in LLM-Based Natural Language Commands Processing for Robot Navigation 29 May 2025 · 0 repositories · arXiv:2506.00075
-
Scalable Complexity Control Facilitates Reasoning Ability of LLMs 29 May 2025 · 0 repositories · arXiv:2505.23013
-
Agent-UniRAG: A Trainable Open-Source LLM Agent Framework for Unified Retrieval-Augmented Generation Systems 28 May 2025 · 0 repositories · arXiv:2505.22571
-
Breaking the Cloak! Unveiling Chinese Cloaked Toxicity with Homophone Graph and Toxic Lexicon 28 May 2025 · 0 repositories · arXiv:2505.22184
-
Climate Finance Bench 28 May 2025 · 1 repository · arXiv:2505.22752
-
Cross-modal RAG: Sub-dimensional Retrieval-Augmented Text-to-Image Generation 28 May 2025 · 1 repository · arXiv:2505.21956
-
Improving QA Efficiency with DistilBERT: Fine-Tuning and Inference on mobile Intel CPUs 28 May 2025 · 0 repositories · arXiv:2505.22937
-
Multi-MLLM Knowledge Distillation for Out-of-Context News Detection 28 May 2025 · 0 repositories · arXiv:2505.22517
-
RAGPPI: RAG Benchmark for Protein-Protein Interactions in Drug Discovery 28 May 2025 · 1 repository · arXiv:2505.23823
-
Say What You Mean: Natural Language Access Control with Large Language Models for Internet of Things 28 May 2025 · 0 repositories · arXiv:2505.23835
-
SkewRoute: Training-Free LLM Routing for Knowledge Graph Retrieval-Augmented Generation via Score Skewness of Retrieved Context 28 May 2025 · 0 repositories · arXiv:2505.23841
-
VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learning 28 May 2025 · 1 repository · arXiv:2505.22019Syntology official: no sample here; runs from other or unrecorded repositories · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample) · 1 pointer-only (licence)
-
Diagnosing and Resolving Cloud Platform Instability with Multi-modal RAG LLMs 27 May 2025 · 0 repositories · arXiv:2505.21419
-
From prosthetic memory to prosthetic denial: Auditing whether large language models are prone to mass atrocity denialism 27 May 2025 · 0 repositories · arXiv:2505.21753
-
Long Context Scaling: Divide and Conquer via Multi-Agent Question-driven Collaboration 27 May 2025 · 0 repositories · arXiv:2505.20625
-
Privacy-Preserving Chest X-ray Report Generation via Multimodal Federated Learning with ViT and GPT-2 27 May 2025 · 0 repositories · arXiv:2505.21715
-
A Theoretical Framework for Grokking: Interpolation followed by Riemannian Norm Minimisation 26 May 2025 · 0 repositories · arXiv:2505.20172
-
Anveshana: A New Benchmark Dataset for Cross-Lingual Information Retrieval On English Queries and Sanskrit Documents 26 May 2025 · 0 repositories · arXiv:2505.19494
-
Automated evaluation of children's speech fluency for low-resource languages 26 May 2025 · 0 repositories · arXiv:2505.19671
-
Benchmarking Multimodal Knowledge Conflict for Large Multimodal Models 26 May 2025 · 1 repository · arXiv:2505.19509
-
Beyond Specialization: Benchmarking LLMs for Transliteration of Indian Languages 26 May 2025 · 0 repositories · arXiv:2505.19851
-
Calibrating Pre-trained Language Classifiers on LLM-generated Noisy Labels via Iterative Refinement 26 May 2025 · 1 repository · arXiv:2505.19675
-
Conversational Lexicography: Querying Lexicographic Data on Knowledge Graphs with SPARQL through Natural Language 26 May 2025 · 0 repositories · arXiv:2505.19971
-
Detection of Suicidal Risk on Social Media: A Hybrid Model 26 May 2025 · 0 repositories · arXiv:2505.23797
-
DGRAG: Distributed Graph-based Retrieval-Augmented Generation in Edge-Cloud Systems 26 May 2025 · 0 repositories · arXiv:2505.19847
-
DoctorRAG: Medical RAG Fusing Knowledge with Patient Analogy through Textual Gradients 26 May 2025 · 0 repositories · arXiv:2505.19538
-
Emotion Classification In-Context in Spanish 26 May 2025 · 0 repositories · arXiv:2505.20571
-
ESLM: Risk-Averse Selective Language Modeling for Efficient Pretraining 26 May 2025 · 0 repositories · arXiv:2505.19893
-
KnowTrace: Bootstrapping Iterative Retrieval-Augmented Generation with Structured Knowledge Tracing 26 May 2025 · 1 repository · arXiv:2505.20245
-
MA-RAG: Multi-Agent Retrieval-Augmented Generation via Collaborative Chain-of-Thought Reasoning 26 May 2025 · 0 repositories · arXiv:2505.20096
-
NeuSym-RAG: Hybrid Neural Symbolic Retrieval with Multiview Structuring for PDF Question Answering 26 May 2025 · 1 repository · arXiv:2505.19754
-
syftr: Pareto-Optimal Generative AI 26 May 2025 · 1 repository · arXiv:2505.20266
-
AI4Math: A Native Spanish Benchmark for University-Level Mathematical Reasoning in Large Language Models 25 May 2025 · 0 repositories · arXiv:2505.18978
-
Conventional Contrastive Learning Often Falls Short: Improving Dense Retrieval with Cross-Encoder Listwise Distillation and Synthetic Data 25 May 2025 · 1 repository · arXiv:2505.19274
-
Hermes@DravidianLangTech 2025: Sentiment Analysis of Dravidian Languages using XLM-RoBERTa 25 May 2025 · 1 repository
-
Hypercube-RAG: Hypercube-Based Retrieval-Augmented Generation for In-domain Scientific Question-Answering 25 May 2025 · 1 repository · arXiv:2505.19288
-
Investigating Pedagogical Teacher and Student LLM Agents: Genetic Adaptation Meets Retrieval Augmented Generation Across Learning Style 25 May 2025 · 0 repositories · arXiv:2505.19173
-
Optimized Text Embedding Models and Benchmarks for Amharic Passage Retrieval 25 May 2025 · 1 repository · arXiv:2505.19356
-
POQD: Performance-Oriented Query Decomposer for Multi-vector retrieval 25 May 2025 · 1 repository · arXiv:2505.19189Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 3 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples) · 5 pointer-only (licence)
-
Retrieval-Augmented Generation for Service Discovery: Chunking Strategies and Benchmarking 25 May 2025 · 0 repositories · arXiv:2505.19310
-
A Survey of LLM × DATA 24 May 2025 · 2 repositories · arXiv:2505.18458
-
Benchmarking Poisoning Attacks against Retrieval-Augmented Generation 24 May 2025 · 0 repositories · arXiv:2505.18543
-
BRIT: Bidirectional Retrieval over Unified Image-Text Graph 24 May 2025 · 0 repositories · arXiv:2505.18450
-
Federated Retrieval-Augmented Generation: A Systematic Mapping Study 24 May 2025 · 0 repositories · arXiv:2505.18906
-
From Reddit to Generative AI: Evaluating Large Language Models for Anxiety Support Fine-tuned on Social Media Data 24 May 2025 · 0 repositories · arXiv:2505.18464
-
GainRAG: Preference Alignment in Retrieval-Augmented Generation through Gain Signal Synthesis 24 May 2025 · 1 repository · arXiv:2505.18710
-
LLMs for Supply Chain Management 24 May 2025 · 0 repositories · arXiv:2505.18597
-
Pruning for Performance: Efficient Idiom and Metaphor Classification in Low-Resource Konkani Using mBERT 24 May 2025 · 0 repositories · arXiv:2506.02005
-
Removal of Hallucination on Hallucination: Debate-Augmented RAG 24 May 2025 · 1 repository · arXiv:2505.18581Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 3 unverified (of 4 harvested samples)
-
Strong Membership Inference Attacks on Massive Datasets and (Moderately) Large Language Models 24 May 2025 · 0 repositories · arXiv:2505.18773
-
The Silent Saboteur: Imperceptible Adversarial Attacks against Black-Box Retrieval-Augmented Generation Systems 24 May 2025 · 0 repositories · arXiv:2505.18583
-
Emergence of Hebbian Dynamics in Regularized Non-Local Learners 23 May 2025 · 0 repositories · arXiv:2505.18069
-
LLM assisted web application functional requirements generation: A case study of four popular LLMs over a Mess Management System 23 May 2025 · 0 repositories · arXiv:2505.18019
-
Multi-Scale Probabilistic Generation Theory: A Hierarchical Framework for Interpreting Large Language Models 23 May 2025 · 0 repositories · arXiv:2505.18244
Tasks archive 2025-07-28
20 shown of 1,523 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
| Task | Papers |
|---|---|
| Language Modelling | 1,799 |
| Language Modeling | 1,406 |
| Retrieval | 1,368 |
| RAG | 1,244 |
| Retrieval-augmented Generation | 1,068 |
| Question Answering | 996 |
| Sentence | 866 |
| Large Language Model | 480 |
| Sentiment Analysis | 458 |
| Text Classification | 448 |
| Text Generation | 417 |
| text-classification | 397 |
| Transfer Learning | 369 |
| Classification | 300 |
| Information Retrieval | 291 |
| Natural Language Understanding | 285 |
| Word Embeddings | 274 |
| Named Entity Recognition | 264 |
| Articles | 255 |
| named-entity-recognition | 254 |
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections