{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cosmos-world-foundation-model-platform-for","title":"Cosmos World Foundation Model Platform for Physical AI","arxiv_id":"2501.03575","date":"2025-01-07","proceeding":null,"authors":["Nvidia",":","Niket Agarwal","Arslan Ali","Maciej Bala","Yogesh Balaji","Erik Barker","Tiffany Cai","Prithvijit Chattopadhyay","Yongxin Chen","Yin Cui","Yifan Ding","Daniel Dworakowski","Jiaojiao Fan","Michele Fenzi","Francesco Ferroni","Sanja Fidler","Dieter Fox","Songwei Ge","Yunhao Ge","Jinwei Gu","Siddharth Gururani","Ethan He","Jiahui Huang","Jacob Huffman","Pooya Jannaty","Jingyi Jin","Seung Wook Kim","Gergely Klár","Grace Lam","Shiyi Lan","Laura Leal-Taixe","Anqi Li","Zhaoshuo Li","Chen-Hsuan Lin","Tsung-Yi Lin","Huan Ling","Ming-Yu Liu","Xian Liu","Alice Luo","Qianli Ma","Hanzi Mao","Kaichun Mo","Arsalan Mousavian","Seungjun Nah","Sriharsha Niverty","David Page","Despoina Paschalidou","Zeeshan Patel","Lindsey Pavao","Morteza Ramezanali","Fitsum Reda","Xiaowei Ren","Vasanth Rao Naik Sabavat","Ed Schmerling","Stella Shi","Bartosz Stefaniak","Shitao Tang","Lyne Tchapmi","Przemek Tredak","Wei-Cheng Tseng","Jibin Varghese","Hao Wang","Haoxiang Wang","Heng Wang","Ting-Chun Wang","Fangyin Wei","Xinyue Wei","Jay Zhangjie Wu","Jiashu Xu","Wei Yang","Lin Yen-Chen","Xiaohui Zeng","Yu Zeng","Jing Zhang","Qinsheng Zhang","Yuxuan Zhang","Qingqing Zhao","Artur Zolkowski"],"abstract":"Physical AI needs to be trained digitally first. It needs a digital twin of itself, the policy model, and a digital twin of the world, the world model. In this paper, we present the Cosmos World Foundation Model Platform to help developers build customized world models for their Physical AI setups. We position a world foundation model as a general-purpose world model that can be fine-tuned into customized world models for downstream applications. Our platform covers a video curation pipeline, pre-trained world foundation models, examples of post-training of pre-trained world foundation models, and video tokenizers. To help Physical AI builders solve the most critical problems of our society, we make Cosmos open-source and our models open-weight with permissive licenses available via https://github.com/nvidia-cosmos/cosmos-predict1.","url_abs":"https://arxiv.org/abs/2501.03575v2","url_pdf":"https://arxiv.org/pdf/2501.03575v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cosmos-world-foundation-model-platform-for","repo_url":"https://github.com/nvidia-cosmos/cosmos-predict1","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"cosmos-world-foundation-model-platform-for","repo_url":"https://github.com/nvidia/cosmos-tokenizer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"cosmos-world-foundation-model-platform-for","repo_url":"https://github.com/nvlabs/tokenbench","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"cosmos-world-foundation-model-platform-for","repo_url":"https://github.com/nvidia-cosmos/cosmos-transfer1","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":null,"task_name":"Position"},{"task_slug":"model","task_name":"model"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2501.03575","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2501.03575"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nvlabs/tokenbench","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nvidia-cosmos/cosmos-transfer1","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nvidia/cosmos-tokenizer","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nvidia-cosmos/cosmos-predict1","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_fixture":1,"unverified":13},"by_repo_kind":{"official":{"samples":14,"ran":1,"repositories":3}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e5947aba1d10885f","entry":"get_1d_sincos_pos_embed_from_grid","repo":"nvidia-cosmos/cosmos-predict1","repo_kind":"official","path":"cosmos_predict1/autoregressive/modules/embedding.py","file_url":"https://github.com/nvidia-cosmos/cosmos-predict1/blob/HEAD/cosmos_predict1/autoregressive/modules/embedding.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e5947aba1d10885f"}},{"code_sha256_prefix":"ef1f7e1ebf6386e6","entry":"LPIPS","repo":"nvlabs/tokenbench","repo_kind":"official","path":"token_bench/metrics_cli.py","file_url":"https://github.com/nvlabs/tokenbench/blob/HEAD/token_bench/metrics_cli.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ef1f7e1ebf6386e6"}},{"code_sha256_prefix":"2ec238b8692bb23a","entry":"PSNR","repo":"nvlabs/tokenbench","repo_kind":"official","path":"token_bench/metrics_cli.py","file_url":"https://github.com/nvlabs/tokenbench/blob/HEAD/token_bench/metrics_cli.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2ec238b8692bb23a"}},{"code_sha256_prefix":"4ea7aadfca86cd09","entry":"SSIM","repo":"nvlabs/tokenbench","repo_kind":"official","path":"token_bench/metrics_cli.py","file_url":"https://github.com/nvlabs/tokenbench/blob/HEAD/token_bench/metrics_cli.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4ea7aadfca86cd09"}},{"code_sha256_prefix":"cc3e6d9b4eee82e0","entry":"batch2time","repo":"nvidia/cosmos-tokenizer","repo_kind":"official","path":"cosmos_tokenizer/modules/utils.py","file_url":"https://github.com/nvidia/cosmos-tokenizer/blob/HEAD/cosmos_tokenizer/modules/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cc3e6d9b4eee82e0"}},{"code_sha256_prefix":"826dcfd43fab7282","entry":"compute_llama3_ffn_hidden_dim","repo":"nvidia-cosmos/cosmos-predict1","repo_kind":"official","path":"cosmos_predict1/autoregressive/modules/mlp.py","file_url":"https://github.com/nvidia-cosmos/cosmos-predict1/blob/HEAD/cosmos_predict1/autoregressive/modules/mlp.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"826dcfd43fab7282"}},{"code_sha256_prefix":"4431273b1abad22f","entry":"create_norm","repo":"nvidia-cosmos/cosmos-predict1","repo_kind":"official","path":"cosmos_predict1/autoregressive/modules/normalization.py","file_url":"https://github.com/nvidia-cosmos/cosmos-predict1/blob/HEAD/cosmos_predict1/autoregressive/modules/normalization.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4431273b1abad22f"}},{"code_sha256_prefix":"aa4f611e6a714f05","entry":"get_vit_config","repo":"nvidia-cosmos/cosmos-predict1","repo_kind":"official","path":"cosmos_predict1/autoregressive/networks/vit.py","file_url":"https://github.com/nvidia-cosmos/cosmos-predict1/blob/HEAD/cosmos_predict1/autoregressive/networks/vit.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"aa4f611e6a714f05"}},{"code_sha256_prefix":"439497162c72f1dc","entry":"precompute_freqs_cis_2d","repo":"nvidia-cosmos/cosmos-predict1","repo_kind":"official","path":"cosmos_predict1/autoregressive/networks/vit.py","file_url":"https://github.com/nvidia-cosmos/cosmos-predict1/blob/HEAD/cosmos_predict1/autoregressive/networks/vit.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"439497162c72f1dc"}},{"code_sha256_prefix":"7b03dd8d6c1862b8","entry":"preprocess_single","repo":"nvlabs/tokenbench","repo_kind":"official","path":"token_bench/fvd.py","file_url":"https://github.com/nvlabs/tokenbench/blob/HEAD/token_bench/fvd.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7b03dd8d6c1862b8"}},{"code_sha256_prefix":"7176ff743ab3af05","entry":"reshape_for_broadcast","repo":"nvidia-cosmos/cosmos-predict1","repo_kind":"official","path":"cosmos_predict1/autoregressive/networks/vit.py","file_url":"https://github.com/nvidia-cosmos/cosmos-predict1/blob/HEAD/cosmos_predict1/autoregressive/networks/vit.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7176ff743ab3af05"}},{"code_sha256_prefix":"abc31b2f673701c7","entry":"scaled_dot_product_attention","repo":"nvidia-cosmos/cosmos-predict1","repo_kind":"official","path":"cosmos_predict1/autoregressive/modules/attention.py","file_url":"https://github.com/nvidia-cosmos/cosmos-predict1/blob/HEAD/cosmos_predict1/autoregressive/modules/attention.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"abc31b2f673701c7"}},{"code_sha256_prefix":"b48fa21d32c1e846","entry":"space2batch","repo":"nvidia/cosmos-tokenizer","repo_kind":"official","path":"cosmos_tokenizer/modules/utils.py","file_url":"https://github.com/nvidia/cosmos-tokenizer/blob/HEAD/cosmos_tokenizer/modules/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b48fa21d32c1e846"}},{"code_sha256_prefix":"e9de9d559844e9ae","entry":"time2batch","repo":"nvidia/cosmos-tokenizer","repo_kind":"official","path":"cosmos_tokenizer/modules/utils.py","file_url":"https://github.com/nvidia/cosmos-tokenizer/blob/HEAD/cosmos_tokenizer/modules/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e9de9d559844e9ae"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}