Methods › General › Learning Rate Schedules › Log Decay
Cyclical Log Annealing as a Learning Rate Scheduler
Log Decay
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Cyclical log annealing make harsh restarts by creating learning rate spikes to escape a local minimum on weight bias fields while retaining the capability to hang onto a low learning rate unlike cosine annealing, which opens up a discussion of if more greedy stochastic optimization schemes could be viable without as high of a risk of overfitting in late stage training. Though log annealing works similarly to cosine annealing, it's properties are different, as it's base learning rate is much more prevalent and much higher, though this can be adjusted by tweaking minimum decay and restart interval multipliers.
Papers archive 2025-07-28
The archive tags no paper with this method.
Tasks archive 2025-07-28
The archive attaches no task to a paper tagged with this method.
Usage over time archive 2025-07-28
No dated papers to chart.
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections