{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-rising-costs-of-training-frontier-ai","title":"The rising costs of training frontier AI models","arxiv_id":"2405.21015","date":"2024-05-31","proceeding":null,"authors":["Ben Cottier","Robi Rahman","Loredana Fattorini","Nestor Maslej","Tamay Besiroglu","David Owen"],"abstract":"The costs of training frontier AI models have grown dramatically in recent years, but there is limited public data on the magnitude and growth of these expenses. This paper develops a detailed cost model to address this gap, estimating training costs using three approaches that account for hardware, energy, cloud rental, and staff expenses. The analysis reveals that the amortized cost to train the most compute-intensive models has grown precipitously at a rate of 2.4x per year since 2016 (90% CI: 2.0x to 2.9x). For key frontier models, such as GPT-4 and Gemini, the most significant expenses are AI accelerator chips and staff costs, each costing tens of millions of dollars. Other notable costs include server components (15-22%), cluster-level interconnect (9-13%), and energy consumption (2-6%). If the trend of growing development costs continues, the largest training runs will cost more than a billion dollars by 2027, meaning that only the most well-funded organizations will be able to finance frontier AI models.","url_abs":"https://arxiv.org/abs/2405.21015v2","url_pdf":"https://arxiv.org/pdf/2405.21015v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"the-rising-costs-of-training-frontier-ai","repo_url":"https://github.com/epoch-research/training-cost-trends","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2405.21015","atlas_url":"https://app.syntology.ai/?focus=2405.21015","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.21015"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/epoch-research/training-cost-trends","reach":{"status":"ok"}}],"summary":{"ran":2,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"19f349fbe7583606","entry":"float_year_to_datetime","repo":"epoch-research/training-cost-trends","repo_kind":"official","path":"cost_analysis_standalone.py","file_url":"https://github.com/epoch-research/training-cost-trends/blob/HEAD/cost_analysis_standalone.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"19f349fbe7583606"}},{"code_sha256_prefix":"c1d2d8f904682468","entry":"geomean","repo":"epoch-research/training-cost-trends","repo_kind":"official","path":"cost_analysis_standalone.py","file_url":"https://github.com/epoch-research/training-cost-trends/blob/HEAD/cost_analysis_standalone.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c1d2d8f904682468"}},{"code_sha256_prefix":"5ce6b4c3b34dbaec","entry":"datetime_to_float_year","repo":"epoch-research/training-cost-trends","repo_kind":"official","path":"cost_analysis_standalone.py","file_url":"https://github.com/epoch-research/training-cost-trends/blob/HEAD/cost_analysis_standalone.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5ce6b4c3b34dbaec"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}