Methods › General › Activation Functions
Activation Functions
The archive carries no description for this collection.
Methods
All 69 methods in this collection, most-tagged first. Year is the archive's introduced_year; the archive stores 2000 when it has none, shown here as “–”. Papers counts distinct papers the archive tags with the method. Click a heading to sort.
| ReLU | – | 10,350 |
| Sigmoid Activation | – | 7,112 |
| Tanh Activation | – | 6,333 |
| Gated Linear Unit | – | 798 |
| PReLU Parameterized ReLU | – | 119 |
| ReLU6 | – | 95 |
| FEM Features Explanation Method | – | 90 |
| Hard Swish | – | 90 |
| RBF Radial Basis Function | – | 62 |
| Maxout | – | 50 |
| ELU Exponential Linear Unit | – | 44 |
| ARiA Adaptive Richard's Curve Weighted Activation | – | 30 |
| SiLU Sigmoid Linear Unit | – | 30 |
| Shifted Softplus | – | 20 |
| Squared ReLU | – | 15 |
| SwiGLU | – | 13 |
| Softsign Activation | – | 12 |
| ASU Amplifying Sine Unit: An Oscillatory Activation Function for Deep Neural Networks to Recover Nonlinear Oscillations Efficiently | – | 9 |
| CReLU | – | 9 |
| KAF Kernel Activation Function | – | 9 |
| Siren Sinusoidal Representation Network | – | 9 |
| modReLU | – | 6 |
| Poly Polynomial | – | 5 |
| TanhExp Tanh Exponential Activation Function | – | 5 |
| DRA Dynamic Range Activator | – | 4 |
| Hard Sigmoid | – | 4 |
| GCU Growing Cosine Unit | – | 3 |
| Hermite Activation Hermite Polynomial Activation | 2019 | 3 |
| RReLU Randomized Leaky Rectified Linear Units | – | 3 |
| Serf | – | 3 |
| m-arcsinh modified arcsinh | – | 3 |
| AHAF Adaptive Hybrid Activation Function | – | 2 |
| CELU Continuously Differentiable Exponential Linear Units | – | 2 |
| ConGater Controllable Gate Adapter | – | 2 |
| DELU | – | 2 |
| E-swish | – | 2 |
| GeGLU | – | 2 |
| Margin ReLU Margin Rectified Linear Unit | – | 2 |
| PAU Padé Activation Units | – | 2 |
| PELU Parametric Exponential Linear Unit | – | 2 |
| Phish Phish: A Novel Hyper-Optimizable Activation Function | – | 2 |
| ReGLU | – | 2 |
| SReLU S-shaped ReLU | – | 2 |
| ANDHRA Module Ajay N’ Daliparthi Hyper Rectified Activation | – | 1 |
| CoLU Collapsing Linear Unit | – | 1 |
| CosLU Cosine Linear Unit | – | 1 |
| ELiSH Exponential Linear Squashing Activation | – | 1 |
| GCLU Gaussian Calibration Linear Unit | – | 1 |
| GoLU Gompertz Linear Unit | – | 1 |
| Gumbel Activation Gumbel Cross Entropy | – | 1 |
| HardELiSH | – | 1 |
| HeLU Hysteresis Rectified Linear Unit | – | 1 |
| LinComb Linear Combination of Activations | – | 1 |
| NAIL OR Logit-space OR-gate (Normalized Approximate Independent Logit) | – | 1 |
| NLSIG nlogistic-sigmoid function | – | 1 |
| NormLinComb Normalized Linear Combination of Activations | – | 1 |
| ReLUN Rectified Linear Unit N | – | 1 |
| SERLU | – | 1 |
| SSFG regularization Stochastically Scaling Features and Gradients Regularization | – | 1 |
| ScaledSoftSign | – | 1 |
| ShiLU Shifted Rectified Linear Unit | – | 1 |
| Smish | – | 1 |
| Smooth Step | – | 1 |
| StarReLU | – | 1 |
| TAAF The Analog Activation Function | – | 1 |
| a-m-lines a-m-parametric lines | – | 1 |
| ASAF Adaptive Spline Activation Function | – | 0 |
| Hardtanh Activation | – | 0 |
| Lecun's Tanh | 1998 | 0 |