Methods › General › Activation Functions › RReLU
Randomized Leaky Rectified Linear Units
RReLU
Introduced by Bing Xu et al. in Empirical Evaluation of Rectified Activations in Convolutional Network
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Randomized Leaky Rectified Linear Units, or RReLU, are an activation function that randomly samples the negative slope for activation values. It was first proposed and used in the Kaggle NDSB Competition. During training, aⱼᵢ is a random number sampled from a uniform distribution U(l, u). Formally:
yⱼᵢ = xⱼᵢ if xⱼᵢ ≥0 yⱼᵢ = aⱼᵢxⱼᵢ if xⱼᵢ < 0
where
αⱼᵢ ∼U(l, u), l < u and l, u ∈[0,1)
In the test phase, we take average of all the aⱼᵢ in training similar to dropout, and thus set aⱼᵢ to (l+u)/2 to get a deterministic result. As suggested by the NDSB competition winner, aⱼᵢ is sampled from U(3, 8).
At test time, we use:
yⱼᵢ = xⱼᵢ/((l+u)/2)
Papers archive 2025-07-28
3 shown of 3, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
Non-Linearities Improve OrigiNet based on Active Imaging for Micro Expression Recognition 16 May 2020 · 0 repositories · arXiv:2005.07991
-
Towards Single-phase Single-stage Detection of Pulmonary Nodules in Chest CT Imaging 16 Jul 2018 · 0 repositories · arXiv:1807.05972
-
Empirical Evaluation of Rectified Activations in Convolutional Network 5 May 2015 · 2 repositories · arXiv:1505.00853
Tasks archive 2025-07-28
6 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
| Task | Papers |
|---|---|
| General Classification | 1 |
| Image Classification | 1 |
| Micro Expression Recognition | 1 |
| Micro-Expression Recognition | 1 |
| Transfer Learning | 1 |
| image-classification | 1 |
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections