{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/beyond-human-level-accuracy-computational","title":"Beyond Human-Level Accuracy: Computational Challenges in Deep Learning","arxiv_id":"1909.01736","date":"2019-09-03","proceeding":null,"authors":["Joel Hestness","Newsha Ardalani","Greg Diamos"],"abstract":"Deep learning (DL) research yields accuracy and product improvements from both model architecture changes and scale: larger data sets and models, and more computation. For hardware design, it is difficult to predict DL model changes. However, recent prior work shows that as dataset sizes grow, DL model accuracy and model size grow predictably. This paper leverages the prior work to project the dataset and model size growth required to advance DL accuracy beyond human-level, to frontier targets defined by machine learning experts. Datasets will need to grow $33$--$971 \\times$, while models will need to grow $6.6$--$456\\times$ to achieve target accuracies. We further characterize and project the computational requirements to train these applications at scale. Our characterization reveals an important segmentation of DL training challenges for recurrent neural networks (RNNs) that contrasts with prior studies of deep convolutional networks. RNNs will have comparatively moderate operational intensities and very large memory footprint requirements. In contrast to emerging accelerator designs, large-scale RNN training characteristics suggest designs with significantly larger memory capacity and on-chip caches.","url_abs":"https://arxiv.org/abs/1909.01736v1","url_pdf":"https://arxiv.org/pdf/1909.01736v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"beyond-human-level-accuracy-computational","repo_url":"https://github.com/baidu-research/catamount","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1909.01736","atlas_url":"https://app.syntology.ai/?focus=1909.01736","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1909.01736"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/baidu-research/catamount","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":9},"by_repo_kind":{"official":{"samples":9,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"80e960c88c6fcddf","entry":"concat","repo":"baidu-research/catamount","repo_kind":"official","path":"catamount/api/ops.py","file_url":"https://github.com/baidu-research/catamount/blob/HEAD/catamount/api/ops.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"80e960c88c6fcddf"}},{"code_sha256_prefix":"62b570cb7bd5111d","entry":"constant","repo":"baidu-research/catamount","repo_kind":"official","path":"catamount/api/ops.py","file_url":"https://github.com/baidu-research/catamount/blob/HEAD/catamount/api/ops.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"62b570cb7bd5111d"}},{"code_sha256_prefix":"d82f883b2803c5fa","entry":"dynamic_stitch","repo":"baidu-research/catamount","repo_kind":"official","path":"catamount/api/ops.py","file_url":"https://github.com/baidu-research/catamount/blob/HEAD/catamount/api/ops.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d82f883b2803c5fa"}},{"code_sha256_prefix":"fc91758885e66190","entry":"getIntSymbolFromString","repo":"baidu-research/catamount","repo_kind":"official","path":"catamount/api/utils.py","file_url":"https://github.com/baidu-research/catamount/blob/HEAD/catamount/api/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fc91758885e66190"}},{"code_sha256_prefix":"1f573759c4cd2a03","entry":"getPositiveIntSymbolFromString","repo":"baidu-research/catamount","repo_kind":"official","path":"catamount/api/utils.py","file_url":"https://github.com/baidu-research/catamount/blob/HEAD/catamount/api/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1f573759c4cd2a03"}},{"code_sha256_prefix":"1e2d38924eb60e6f","entry":"getSymbolicMaximum","repo":"baidu-research/catamount","repo_kind":"official","path":"catamount/api/utils.py","file_url":"https://github.com/baidu-research/catamount/blob/HEAD/catamount/api/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1e2d38924eb60e6f"}},{"code_sha256_prefix":"f516913bc8e4e27a","entry":"loadGraph","repo":"baidu-research/catamount","repo_kind":"official","path":"catamount/graph/saver.py","file_url":"https://github.com/baidu-research/catamount/blob/HEAD/catamount/graph/saver.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f516913bc8e4e27a"}},{"code_sha256_prefix":"8b5e31c682caed53","entry":"load_tf_session","repo":"baidu-research/catamount","repo_kind":"official","path":"catamount/frameworks/tensorflow.py","file_url":"https://github.com/baidu-research/catamount/blob/HEAD/catamount/frameworks/tensorflow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8b5e31c682caed53"}},{"code_sha256_prefix":"1c15c5792a27a237","entry":"tf_shape_to_catamount","repo":"baidu-research/catamount","repo_kind":"official","path":"catamount/frameworks/tensorflow.py","file_url":"https://github.com/baidu-research/catamount/blob/HEAD/catamount/frameworks/tensorflow.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1c15c5792a27a237"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}