{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deepobs-a-deep-learning-optimizer-benchmark-1","title":"DeepOBS: A Deep Learning Optimizer Benchmark Suite","arxiv_id":"1903.05499","date":"2019-03-13","proceeding":"ICLR 2019 5","authors":["Frank Schneider","Lukas Balles","Philipp Hennig"],"abstract":"Because the choice and tuning of the optimizer affects the speed, and\nultimately the performance of deep learning, there is significant past and\nrecent research in this area. Yet, perhaps surprisingly, there is no generally\nagreed-upon protocol for the quantitative and reproducible evaluation of\noptimization strategies for deep learning. We suggest routines and benchmarks\nfor stochastic optimization, with special focus on the unique aspects of deep\nlearning, such as stochasticity, tunability and generalization. As the primary\ncontribution, we present DeepOBS, a Python package of deep learning\noptimization benchmarks. The package addresses key challenges in the\nquantitative assessment of stochastic optimizers, and automates most steps of\nbenchmarking. The library includes a wide and extensible set of ready-to-use\nrealistic optimization problems, such as training Residual Networks for image\nclassification on ImageNet or character-level language prediction models, as\nwell as popular classics like MNIST and CIFAR-10. The package also provides\nrealistic baseline results for the most popular optimizers on these test\nproblems, ensuring a fair comparison to the competition when benchmarking new\noptimizers, and without having to run costly experiments. It comes with output\nback-ends that directly produce LaTeX code for inclusion in academic\npublications. It supports TensorFlow and is available open source.","url_abs":"http://arxiv.org/abs/1903.05499v1","url_pdf":"http://arxiv.org/pdf/1903.05499v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deepobs-a-deep-learning-optimizer-benchmark-1","repo_url":"https://github.com/fsschneider/deepobs","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"stochastic-optimization","task_name":"Stochastic Optimization"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1903.05499","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1903.05499"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/fsschneider/deepobs","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":2},"by_repo_kind":{"official":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c49458a8b7c9448e","entry":"beautify_lr_sensitivity","repo":"fsschneider/deepobs","repo_kind":"official","path":"deepobs/analyzer/analyze_utils.py","file_url":"https://github.com/fsschneider/deepobs/blob/HEAD/deepobs/analyzer/analyze_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c49458a8b7c9448e"}},{"code_sha256_prefix":"128631bff215ca54","entry":"beautify_plot_performance","repo":"fsschneider/deepobs","repo_kind":"official","path":"deepobs/analyzer/analyze_utils.py","file_url":"https://github.com/fsschneider/deepobs/blob/HEAD/deepobs/analyzer/analyze_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"128631bff215ca54"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}