{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/reducing-bert-pre-training-time-from-3-days","title":"Large Batch Optimization for Deep Learning: Training BERT in 76 minutes","arxiv_id":"1904.00962","date":"2019-04-01","proceeding":"ICLR 2020 1","authors":["Yang You","Jing Li","Sashank Reddi","Jonathan Hseu","Sanjiv Kumar","Srinadh Bhojanapalli","Xiaodan Song","James Demmel","Kurt Keutzer","Cho-Jui Hsieh"],"abstract":"Training large deep neural networks on massive datasets is computationally very challenging. There has been recent surge in interest in using large batch stochastic optimization methods to tackle this issue. The most prominent algorithm in this line of research is LARS, which by employing layerwise adaptive learning rates trains ResNet on ImageNet in a few minutes. However, LARS performs poorly for attention models like BERT, indicating that its performance gains are not consistent across tasks. In this paper, we first study a principled layerwise adaptation strategy to accelerate training of deep neural networks using large mini-batches. Using this strategy, we develop a new layerwise adaptive large batch optimization technique called LAMB; we then provide convergence analysis of LAMB as well as LARS, showing convergence to a stationary point in general nonconvex settings. Our empirical results demonstrate the superior performance of LAMB across various tasks such as BERT and ResNet-50 training with very little hyperparameter tuning. In particular, for BERT training, our optimizer enables use of very large batch sizes of 32868 without any degradation of performance. By increasing the batch size to the memory limit of a TPUv3 Pod, BERT training time can be reduced from 3 days to just 76 minutes (Table 1). The LAMB implementation is available at https://github.com/tensorflow/addons/blob/master/tensorflow_addons/optimizers/lamb.py","url_abs":"https://arxiv.org/abs/1904.00962v5","url_pdf":"https://arxiv.org/pdf/1904.00962v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/tensorflow/addons","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/tensorflow/addons/blob/master/tensorflow_addons/optimizers/lamb.py","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":null},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/Ankur3107/awesome-daily-blog","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/ShadenSmith/ghpages-test","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/Smerity/pytorch-lamb","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/bojone/tiger","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/btahir/tensorflow-LAMB","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/columbia-ai-robotics/semantic-abstraction","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/cybertronai/pytorch-lamb","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/dimakarp1996/LAMB-keras","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/fastalgo/imagenet_resnet50_lamb","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/frgfm/Holocron","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/hannibal046/pluglm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/huchen365/ds","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/huggingface/pytorch-image-models","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/jiaowoguanren0615/MambaVision","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/jxbz/fromage","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/kaushaltrivedi/fast-bert","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/liuqiangict/lamb_optimizer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/meltnur/speed","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/skyday123/pytorch-lamb","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/utterworks/fast-bert","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/ymcui/LAMB_Optimizer_TF","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/zaradana/Fast_BERT","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/zhuchen03/maxva","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/AlanFokCo/compensation-tools","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/MindCode-4/code-12/tree/main/Large-Batch-Optimization-for-Deep-Learning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/MindCode-4/code-12/tree/main/gaussian_adaptive_attention","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/MindCode-4/code-7/tree/main/Large-Batch-Optimization-for-Deep-Learning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/MindSpore-scientific-2/code-12/tree/main/Large-Batch-Optimization-for-Deep-Learning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/MindSpore-scientific-2/code-12/tree/main/gaussian_adaptive_attention","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"reducing-bert-pre-training-time-from-3-days","repo_url":"https://github.com/PaddlePaddle/FleetX/blob/develop/examples/resnet/train_fleet_lamb.py","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"paddle","reach":null}],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"stochastic-optimization","task_name":"Stochastic Optimization"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"bert","method_name":"BERT"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"lamb","method_name":"LAMB"},{"method_slug":"lars","method_name":"LARS"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-linear-decay","method_name":"Linear Warmup With Linear Decay"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"weight-decay","method_name":"Weight Decay"},{"method_slug":"wordpiece","method_name":"WordPiece"}],"datasets_introduced":[],"methods_introduced":[{"slug":"lamb","name":"LAMB","full_name":"LAMB"}],"results":[{"leaderboard":"/sota/question-answering-on-squad11-dev","task":"Question Answering","dataset":"SQuAD1.1 dev","model":"BERT large (LAMB optimizer)","rank_in_archive_order":53,"of":55,"metrics":{"F1":"90.584"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1904.00962","atlas_url":"https://app.syntology.ai/?focus=1904.00962","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1904.00962"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/huggingface/pytorch-image-models","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/liuqiangict/lamb_optimizer","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/cybertronai/pytorch-lamb","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hannibal046/pluglm","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/meltnur/speed","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ymcui/LAMB_Optimizer_TF","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/frgfm/Holocron","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Ankur3107/awesome-daily-blog","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jxbz/fromage","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/skyday123/pytorch-lamb","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tensorflow/addons","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ShadenSmith/ghpages-test","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/btahir/tensorflow-LAMB","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MindSpore-scientific-2/code-12/tree/main/Large-Batch-Optimization-for-Deep-Learning","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MindCode-4/code-12/tree/main/gaussian_adaptive_attention","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Smerity/pytorch-lamb","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MindCode-4/code-12/tree/main/Large-Batch-Optimization-for-Deep-Learning","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MindCode-4/code-7/tree/main/Large-Batch-Optimization-for-Deep-Learning","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/PaddlePaddle/FleetX/blob/develop/examples/resnet/train_fleet_lamb.py","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MindSpore-scientific-2/code-12/tree/main/gaussian_adaptive_attention","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/columbia-ai-robotics/semantic-abstraction","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tensorflow/addons/blob/master/tensorflow_addons/optimizers/lamb.py","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/huchen365/ds","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jiaowoguanren0615/MambaVision","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kaushaltrivedi/fast-bert","reach":null}],"summary":{"ran":6,"unverified":5},"by_repo_kind":{"listed":{"samples":11,"ran":6,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"97777449b62c0a36","entry":"get_lr_from_config","repo":"meltnur/speed","repo_kind":"listed","path":"deepspeed/pt/deepspeed_lr_schedules.py","file_url":"https://github.com/meltnur/speed/blob/HEAD/deepspeed/pt/deepspeed_lr_schedules.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"97777449b62c0a36"}},{"code_sha256_prefix":"8ecbd0d0bd8ec80f","entry":"get_scalar_param","repo":"meltnur/speed","repo_kind":"listed","path":"deepspeed/pt/deepspeed_config.py","file_url":"https://github.com/meltnur/speed/blob/HEAD/deepspeed/pt/deepspeed_config.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"8ecbd0d0bd8ec80f"}},{"code_sha256_prefix":"77529b2a36ec2e49","entry":"parse_resource_filter","repo":"meltnur/speed","repo_kind":"listed","path":"deepspeed/pt/deepspeed_run.py","file_url":"https://github.com/meltnur/speed/blob/HEAD/deepspeed/pt/deepspeed_run.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"77529b2a36ec2e49"}},{"code_sha256_prefix":"6a8666be8c9e0af6","entry":"piecewise_linear","repo":"bojone/tiger","repo_kind":"listed","path":"tiger.py","file_url":"https://github.com/bojone/tiger/blob/HEAD/tiger.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6a8666be8c9e0af6"}},{"code_sha256_prefix":"7ced0d41eac533bc","entry":"root_mean_square","repo":"bojone/tiger","repo_kind":"listed","path":"tiger.py","file_url":"https://github.com/bojone/tiger/blob/HEAD/tiger.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7ced0d41eac533bc"}},{"code_sha256_prefix":"6b5d363ab4dbf012","entry":"to_python_float","repo":"meltnur/speed","repo_kind":"listed","path":"deepspeed/pt/loss_scaler.py","file_url":"https://github.com/meltnur/speed/blob/HEAD/deepspeed/pt/loss_scaler.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"6b5d363ab4dbf012"}},{"code_sha256_prefix":"a9c0a909a533ca74","entry":"add_tuning_arguments","repo":"meltnur/speed","repo_kind":"listed","path":"deepspeed/pt/deepspeed_lr_schedules.py","file_url":"https://github.com/meltnur/speed/blob/HEAD/deepspeed/pt/deepspeed_lr_schedules.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"a9c0a909a533ca74"}},{"code_sha256_prefix":"532b9a790bd9609b","entry":"fetch_hostfile","repo":"meltnur/speed","repo_kind":"listed","path":"deepspeed/pt/deepspeed_run.py","file_url":"https://github.com/meltnur/speed/blob/HEAD/deepspeed/pt/deepspeed_run.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"532b9a790bd9609b"}},{"code_sha256_prefix":"6e89a38d49e658cc","entry":"get_config_from_args","repo":"meltnur/speed","repo_kind":"listed","path":"deepspeed/pt/deepspeed_lr_schedules.py","file_url":"https://github.com/meltnur/speed/blob/HEAD/deepspeed/pt/deepspeed_lr_schedules.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"6e89a38d49e658cc"}},{"code_sha256_prefix":"93f7995d74994b25","entry":"get_fp16_enabled","repo":"meltnur/speed","repo_kind":"listed","path":"deepspeed/pt/deepspeed_config.py","file_url":"https://github.com/meltnur/speed/blob/HEAD/deepspeed/pt/deepspeed_config.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"93f7995d74994b25"}},{"code_sha256_prefix":"3c0009bff557332d","entry":"get_loss_scale","repo":"meltnur/speed","repo_kind":"listed","path":"deepspeed/pt/deepspeed_config.py","file_url":"https://github.com/meltnur/speed/blob/HEAD/deepspeed/pt/deepspeed_config.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"3c0009bff557332d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}