Methods › General › Regularization

Regularization

58 methods 30,804 papers tagged archive 2025-07-28

Regularization strategies are designed to reduce the test error of a machine learning algorithm, possibly at the expense of training error. Many different forms of regularization exist in the field of deep learning. Below you can find a constantly updating list of regularization strategies.

Methods

All 58 methods in this collection, most-tagged first. Year is the archive's introduced_year; the archive stores 2000 when it has none, shown here as “–”. Papers counts distinct papers the archive tags with the method. Click a heading to sort.


Dropout	–	27,472
Label Smoothing	1985	14,327
Attention Dropout	2018	10,892
Weight Decay	1943	10,713
Entropy Regularization	–	1,128
R1 Regularization	–	518
Early Stopping	1995	468
Stochastic Depth	–	463
Path Length Regularization	–	182
Variational Dropout	–	141
DropBlock	–	132
Off-Diagonal Orthogonal Regularization	–	132
Target Policy Smoothing	–	116
L1 Regularization	1986	90
DropConnect	–	84
ALS Adaptive Label Smoothing	–	81
Embedding Dropout	–	64
PGM Probability Guided Maxout	–	63
Activation Regularization	–	56
Temporal Activation Regularization	–	53
LCC Lipschitz Constant Constraint	–	33
Zoneout	–	29
SpatialDropout	–	28
Orthogonal Regularization	–	27
Manifold Mixup	–	26
DropPath	–	22
SCN Self-Cure Network	–	18
SRN Stable Rank Normalization	–	17
GAN Feature Matching	–	16
Adaptive Dropout	–	15
Concrete Dropout	–	14
Auxiliary Batch Normalization	–	12
LayerScale	–	12
Euclidean Norm Regularization	–	9
FIERCE Feature Information Entropy Regularized Cross Entropy	–	7
Discriminative Regularization	–	6
GMVAE Gaussian Mixture Variational Autoencoder	–	5
SVD Parameterization Singular Value Decomposition Parameterization	–	5
Shake-Shake Regularization	–	5
Temporal Dropout Temporal Dropout or TempD	–	5
LVR Low Variance Regularization	–	4
LayerDrop	–	4
Sensor Dropout Sensor Dropout or SensD	–	4
Batch Nuclear-norm Maximization	–	3
GradDrop Gradient Sign Dropout	–	3
Recurrent Dropout	–	3
ShakeDrop	–	3
Early Dropout	–	2
Fraternal Dropout	–	2
ScheduledDropPath	–	2
Targeted Dropout	–	2
AutoDropout	–	1
Band Dropout	–	1
DropPathway	–	1
STTP Spectral Tensor Train Parameterization	–	1
Spectral Dropout	–	1
Weights Reset	–	1
Checkerboard Dropout	–	0