Methods › General › Large Batch Optimization › LARS
LARS
Introduced by Yang You et al. in Large Batch Training of Convolutional Networks
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Layer-wise Adaptive Rate Scaling, or LARS, is a large batch optimization technique. There are two notable differences between LARS and other adaptive algorithms such as Adam or RMSProp: first, LARS uses a separate learning rate for each layer and not for each weight. And second, the magnitude of the update is controlled with respect to the weight norm for better control of training speed.
mₜ = β₁mₜ₋₁ + (1-β₁)(gₜ + λxₜ) xₜ₊₁⁽ⁱ⁾ = xₜ⁽ⁱ⁾ - ηₜ(ϕ(|| xₜ⁽ⁱ⁾ ||))/(|| mₜ⁽ⁱ⁾ ||)mₜ⁽ⁱ⁾
In Syntology about another paper, arXiv:2308.09618: repository known, no verified implementation (declared_code; repositories: 2 listed of 2 in the graph, strongest link: official). The lookup matched a paper by its title, arXiv:2308.09618, not a method entry. About another paper: this answer is about arXiv:2308.09618, while the archive credits arXiv:1708.03888.
From the lookup's note on this name: The same name also belongs to “Layer-wise Adaptive Rate Scaling (LARS)”, a method in the paper “Large Batch Training of Convolutional Networks” (arXiv:1708.03888, 977 citations), which the lookup finds cited far more than the paper this answer is about (citations not recorded), so this answer may be about a different sense of the name. For arXiv:1708.03888: its page on this site · ask the lookup about it.
Origin paper of this answer, arXiv:2308.09618 (its page on this site · its page on Syntology).
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 77 · with a code link: 34 · where Syntology ran a sample: 16 (13 with a run with no instrument failure, 3 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (16 of 77 tagged: 13 with a run with no instrument failure, 3 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 77 of 77, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
Analyzing Breast Cancer Survival Disparities by Race and Demographic Location: A Survival Analysis Approach 8 Jun 2025 · 0 repositories · arXiv:2506.07191
-
Enhancing Federated Survival Analysis through Peer-Driven Client Reputation in Healthcare 22 May 2025 · 0 repositories · arXiv:2505.16190
-
Is Self-Supervised Pre-training on Satellite Imagery Better than ImageNet? A Systematic Study with Sentinel-2 15 Feb 2025 · 0 repositories · arXiv:2502.10669
-
Self-Supervised Frameworks for Speaker Verification via Bootstrapped Positive Sampling 29 Jan 2025 · 1 repository · arXiv:2501.17772
-
Tackling Small Sample Survival Analysis via Transfer Learning: A Study of Colorectal Cancer Prognosis 21 Jan 2025 · 1 repository · arXiv:2501.12421
-
Prediction of Lung Metastasis from Hepatocellular Carcinoma using the SEER Database 20 Jan 2025 · 1 repository · arXiv:2501.11720
-
Diagnosis and Severity Assessment of Ulcerative Colitis using Self Supervised Learning 9 Dec 2024 · 0 repositories · arXiv:2412.07806
-
SEER: Self-Aligned Evidence Extraction for Retrieval-Augmented Generation 15 Oct 2024 · 0 repositories · arXiv:2410.11315
-
The Informed Elastic Net for Fast Grouped Variable Selection and FDR Control in Genomics Research 7 Oct 2024 · 0 repositories · arXiv:2410.05211
-
Do Not Design, Learn: A Trainable Scoring Function for Uncertainty Estimation in Generative LLMs 17 Jun 2024 · 0 repositories · arXiv:2406.11278
-
Efficient Preference-based Reinforcement Learning via Aligned Experience Estimation 29 May 2024 · 0 repositories · arXiv:2405.18688
-
Self-Supervised Multiple Instance Learning for Acute Myeloid Leukemia Classification 8 Mar 2024 · 0 repositories · arXiv:2403.05379
-
SEER: Facilitating Structured Reasoning and Explanation via Reinforcement Learning 24 Jan 2024 · 1 repository · arXiv:2401.13246Syntology official (archive's flag): 12 ran · 13 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 2 violated, 4 with no contract checked; 7 where Syntology's instrument failed) · 3 unverified (of 16 harvested samples) · 2 pointer-only (licence)
-
Survival Analysis of Young Triple-Negative Breast Cancer Patients 15 Jan 2024 · 0 repositories · arXiv:2401.08712
-
Enhancing Contrastive Learning with Efficient Combinatorial Positive Pairing 11 Jan 2024 · 0 repositories · arXiv:2401.05730
-
Quantum Algorithms for the Pathwise Lasso 21 Dec 2023 · 0 repositories · arXiv:2312.14141
-
Feature Extraction for Generative Medical Imaging Evaluation: New Evidence Against an Evolving Trend 22 Nov 2023 · 2 repositories · arXiv:2311.13717
-
SEER : A Knapsack approach to Exemplar Selection for In-Context HybridQA 10 Oct 2023 · 1 repository · arXiv:2310.06675Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 10 unverified (of 16 harvested samples)
-
Revisiting LARS for Large Batch Training Generalization of Neural Networks 25 Sep 2023 · 0 repositories · arXiv:2309.14053
-
Accelerating Large Batch Training via Gradient Signal to Noise Ratio (GSNR) 24 Sep 2023 · 0 repositories · arXiv:2309.13681
-
SEER: Super-Optimization Explorer for HLS using E-graph Rewriting with MLIR 15 Aug 2023 · 0 repositories · arXiv:2308.07654
-
The Role of Entropy and Reconstruction in Multi-View Self-Supervised Learning 20 Jul 2023 · 1 repository · arXiv:2307.10907Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
Semi-supervised learning made simple with self-supervised clustering 13 Jun 2023 · 1 repository · arXiv:2306.07483Syntology official (archive's flag): 2 ran · 2 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 1 where Syntology's instrument failed) · 3 unverified (of 5 harvested samples) · 5 pointer-only (licence)
-
Hiding in Plain Sight: Disguising Data Stealing Attacks in Federated Learning 5 Jun 2023 · 2 repositories · arXiv:2306.03013
-
Using Spatio-Temporal Dual-Stream Network with Self-Supervised Learning for Lung Tumor Classification on Radial Probe Endobronchial Ultrasound Video 4 May 2023 · 0 repositories · arXiv:2305.02719
-
In-Distribution and Out-of-Distribution Self-supervised ECG Representation Learning for Arrhythmia Detection 13 Apr 2023 · 0 repositories · arXiv:2304.06427
-
Saddle-to-Saddle Dynamics in Diagonal Linear Networks 2 Apr 2023 · 0 repositories · arXiv:2304.00488
-
Towards domain-invariant Self-Supervised Learning with Batch Styles Standardization 10 Mar 2023 · 0 repositories · arXiv:2303.06088
-
InfoBatch: Lossless Training Speed Up by Unbiased Dynamic Data Pruning 8 Mar 2023 · 1 repository · arXiv:2303.04947Syntology official: harvested, nothing ran · 0 ran · 2 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
Towards a Unified Theoretical Understanding of Non-contrastive Learning via Rank Differential Mechanism 4 Mar 2023 · 1 repository · arXiv:2303.02387Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)
-
Perfect is the enemy of test oracle 3 Feb 2023 · 1 repository · arXiv:2302.01488
-
Leveraging the Third Dimension in Contrastive Learning 27 Jan 2023 · 0 repositories · arXiv:2301.11790
-
A Simple Recipe for Competitive Low-compute Self supervised Vision Models 23 Jan 2023 · 0 repositories · arXiv:2301.09451
-
Joint Self-Supervised Image-Volume Representation Learning with Intra-Inter Contrastive Clustering 4 Dec 2022 · 0 repositories · arXiv:2212.01893
-
How Well Do Unsupervised Learning Algorithms Model Human Real-time and Life-long Learning? 1 Nov 2022 · 1 repository
-
Large-batch Optimization for Dense Visual Predictions 20 Oct 2022 · 1 repository · arXiv:2210.11078
-
The Hidden Uniform Cluster Prior in Self-Supervised Learning 13 Oct 2022 · 1 repository · arXiv:2210.07277
-
Effective Self-supervised Pre-training on Low-compute Networks without Distillation 6 Oct 2022 · 1 repository · arXiv:2210.02808Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified (of 9 harvested samples) · 3 pointer-only (licence)
-
A Formal Comparison between Datalog-based Languages for Stream Reasoning (extended version) 26 Aug 2022 · 0 repositories · arXiv:2208.12726
-
Communication-efficient Distributed Learning for Large Batch Optimization 17 Jul 2022 · 1 repository
-
StudioGAN: A Taxonomy and Benchmark of GANs for Image Synthesis 19 Jun 2022 · 2 repositories · arXiv:2206.09479Syntology official (archive's flag): 1 ran · 8 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 2 where Syntology's instrument failed) · 3 unverified (of 11 harvested samples) · 2 pointer-only (licence)
-
Evaluation of Contrastive Learning with Various Code Representations for Code Clone Detection 17 Jun 2022 · 0 repositories · arXiv:2206.08726
-
Spatial Cross-Attention Improves Self-Supervised Visual Representation Learning 7 Jun 2022 · 0 repositories · arXiv:2206.05028
-
TTAPS: Test-Time Adaption by Aligning Prototypes using Self-Supervision 18 May 2022 · 1 repository · arXiv:2205.08731
-
Chasing Streams with Existential Rules 4 May 2022 · 0 repositories · arXiv:2205.02220
-
auton-survival: an Open-Source Package for Regression, Counterfactual Estimation, Evaluation and Phenotyping with Censored Time-to-Event Data 15 Apr 2022 · 3 repositories · arXiv:2204.07276
-
Unsupervised Salient Object Detection with Spectral Cluster Voting 23 Mar 2022 · 1 repository · arXiv:2203.12614Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 5 where Syntology's instrument failed) · 4 unverified (of 9 harvested samples) · 4 pointer-only (licence)
-
Towards Self-Supervised Gaze Estimation 21 Mar 2022 · 0 repositories · arXiv:2203.10974
-
Influence of different factors on survival of patients with colorectal cancer 5 Feb 2022 · 0 repositories · arXiv:2202.03425
-
SurvTRACE: Transformers for Survival Analysis with Competing Events 2 Oct 2021 · 1 repository · arXiv:2110.00855Syntology official (archive's flag): 7 ran · 7 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 2 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 9 harvested samples) · 2 pointer-only (licence)
-
Contrastive Clustering to Mine Pseudo Parallel Data for Unsupervised Translation 29 Sep 2021 · 0 repositories
-
PASS: An ImageNet replacement for self-supervised pretraining without humans 27 Sep 2021 · 1 repository · arXiv:2109.13228Syntology official: no sample here; runs from other or unrecorded repositories · 11 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 2 honoured, 2 violated, 4 with no contract checked; 3 where Syntology's instrument failed) · 3 unverified (of 14 harvested samples) · 9 pointer-only (licence)
-
Resource and data efficient self supervised learning 3 Sep 2021 · 0 repositories · arXiv:2109.01721
-
How Transferable Are Self-supervised Features in Medical Image Classification Tasks? 23 Aug 2021 · 0 repositories · arXiv:2108.10048
-
Distributed Deep Learning in Open Collaborations 18 Jun 2021 · 2 repositories · arXiv:2106.10207Syntology official: harvested, nothing ran · 0 ran · 5 unverified (of 5 harvested samples)
-
Faster than LASER -- Towards Stream Reasoning with Deep Neural Networks 15 Jun 2021 · 0 repositories · arXiv:2106.08457
-
Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples 28 Apr 2021 · 4 repositories · arXiv:2104.13963Syntology official (archive's flag): 2 ran · 19 ran (of which 4 constructed an object rather than computing a result; 16 with no instrument failure: 0 honoured, 0 violated, 16 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 21 harvested samples) · 3 pointer-only (licence)
-
Prediction of Prognosis and Survival of Patients with Gastric Cancer by Weighted Improved Random Forest Model 10 Apr 2021 · 0 repositories
-
Self-Supervised Training Enhances Online Continual Learning 25 Mar 2021 · 0 repositories · arXiv:2103.14010
-
Efficient Visual Pretraining with Contrastive Detection 19 Mar 2021 · 2 repositories · arXiv:2103.10957
-
Improving Computational Efficiency in Visual Reinforcement Learning via Stored Embeddings 4 Mar 2021 · 1 repository · arXiv:2103.02886
-
Self-supervised Pretraining of Visual Features in the Wild 2 Mar 2021 · 1 repository · arXiv:2103.01988
-
A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes 12 Feb 2021 · 0 repositories · arXiv:2102.06356
-
Evaluating Deep Learning in SystemML using Layer-wise Adaptive Rate Scaling(LARS) Optimizer 5 Feb 2021 · 0 repositories · arXiv:2102.03018
-
Fast Training of Contrastive Learning with Intermediate Contrastive Loss 1 Jan 2021 · 0 repositories
-
Study on the Large Batch Size Training of Neural Networks Based on the Second Order Gradient 16 Dec 2020 · 0 repositories · arXiv:2012.08795
-
Improving Layer-wise Adaptive Rate Methods using Trust Ratio Clipping 27 Nov 2020 · 0 repositories · arXiv:2011.13584
-
Self-Supervised Ranking for Representation Learning 14 Oct 2020 · 0 repositories · arXiv:2010.07258
-
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments 17 Jun 2020 · 18 repositories · arXiv:2006.09882Syntology community repositories only · 13 ran (of which 4 constructed an object rather than computing a result; 6 with no instrument failure: 1 honoured, 1 violated, 4 with no contract checked; 7 where Syntology's instrument failed) · 4 unverified (of 17 harvested samples) · 6 pointer-only (licence)
-
Supervised Contrastive Learning 23 Apr 2020 · 26 repositories · arXiv:2004.11362Syntology community repositories only · 17 ran (of which 0 constructed an object rather than computing a result; 12 with no instrument failure: 2 honoured, 0 violated, 10 with no contract checked; 5 where Syntology's instrument failed) · 6 unverified (of 23 harvested samples) · 5 pointer-only (licence)
-
A Simple Framework for Contrastive Learning of Visual Representations 13 Feb 2020 · 96 repositories · arXiv:2002.05709Syntology official (archive's flag): 2 ran · 115 ran (of which 33 constructed an object rather than computing a result; 89 with no instrument failure: 1 honoured, 1 violated, 87 with no contract checked; 26 where Syntology's instrument failed) · 22 unverified (of 137 harvested samples) · 52 pointer-only (licence)
-
Large Batch Optimization for Deep Learning: Training BERT in 76 minutes 1 Apr 2019 · 32 repositories · arXiv:1904.00962Syntology community repositories only · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 5 unverified (of 11 harvested samples) · 9 pointer-only (licence)
-
SNAP: A semismooth Newton algorithm for pathwise optimization with optimal local convergence rate and oracle properties 9 Oct 2018 · 0 repositories · arXiv:1810.03814
-
Convergence Analysis of Gradient Descent Algorithms with Proportional Updates 9 Jan 2018 · 0 repositories · arXiv:1801.03137
-
Large Batch Training of Convolutional Networks with Layer-wise Adaptive Rate Scaling 1 Jan 2018 · 0 repositories
-
ImageNet Training in Minutes 14 Sep 2017 · 1 repository · arXiv:1709.05011
-
Large Batch Training of Convolutional Networks 13 Aug 2017 · 12 repositories · arXiv:1708.03888Syntology 4 ran (of which 4 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified; every one of the 4 samples that ran constructed an object rather than computing a result (of 5 harvested samples) · 1 pointer-only (licence)
Tasks archive 2025-07-28
20 shown of 106 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections