{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/community-research-earth-digital-intelligence","title":"Community Research Earth Digital Intelligence Twin (CREDIT)","arxiv_id":"2411.07814","date":"2024-11-09","proceeding":null,"authors":["John Schreck","Yingkai Sha","William Chapman","Dhamma Kimpara","Judith Berner","Seth McGinnis","Arnold Kazadi","Negin Sobhani","Ben Kirk","David John Gagne II"],"abstract":"Recent advancements in artificial intelligence (AI) for numerical weather prediction (NWP) have significantly transformed atmospheric modeling. AI NWP models outperform traditional physics-based systems, such as the Integrated Forecast System (IFS), across several global metrics while requiring fewer computational resources. However, existing AI NWP models face limitations related to training datasets and timestep choices, often resulting in artifacts that reduce model performance. To address these challenges, we introduce the Community Research Earth Digital Intelligence Twin (CREDIT) framework, developed at NSF NCAR. CREDIT provides a flexible, scalable, and user-friendly platform for training and deploying AI-based atmospheric models on high-performance computing systems. It offers an end-to-end pipeline for data preprocessing, model training, and evaluation, democratizing access to advanced AI NWP capabilities. We demonstrate CREDIT's potential through WXFormer, a novel deterministic vision transformer designed to predict atmospheric states autoregressively, addressing common AI NWP issues like compounding error growth with techniques such as spectral normalization, padding, and multi-step training. Additionally, to illustrate CREDIT's flexibility and state-of-the-art model comparisons, we train the FUXI architecture within this framework. Our findings show that both FUXI and WXFormer, trained on six-hourly ERA5 hybrid sigma-pressure levels, generally outperform IFS HRES in 10-day forecasts, offering potential improvements in efficiency and forecast accuracy. CREDIT's modular design enables researchers to explore various models, datasets, and training configurations, fostering innovation within the scientific community.","url_abs":"https://arxiv.org/abs/2411.07814v1","url_pdf":"https://arxiv.org/pdf/2411.07814v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"community-research-earth-digital-intelligence","repo_url":"https://github.com/ncar/credit-arxiv","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"community-research-earth-digital-intelligence","repo_url":"https://github.com/ncar/miles-credit","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"vision-transformer","method_name":"Vision Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2411.07814","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2411.07814"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ncar/miles-credit","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ncar/credit-arxiv","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_honours":1,"ran_violates":2,"ran_draft_wrong":1,"unverified":19},"by_repo_kind":{"official":{"samples":23,"ran":4,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d229662b12e581b6","entry":"cast_tuple","repo":"ncar/miles-credit","repo_kind":"official","path":"credit/diffusion_utils.py","file_url":"https://github.com/ncar/miles-credit/blob/HEAD/credit/diffusion_utils.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d229662b12e581b6"}},{"code_sha256_prefix":"60fff7c3c400d7ff","entry":"default","repo":"ncar/miles-credit","repo_kind":"official","path":"credit/attend.py","file_url":"https://github.com/ncar/miles-credit/blob/HEAD/credit/attend.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"60fff7c3c400d7ff"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"ncar/miles-credit","repo_kind":"official","path":"credit/attend.py","file_url":"https://github.com/ncar/miles-credit/blob/HEAD/credit/attend.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"52e10a7d6b0fa036","entry":"once","repo":"ncar/miles-credit","repo_kind":"official","path":"credit/attend.py","file_url":"https://github.com/ncar/miles-credit/blob/HEAD/credit/attend.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"52e10a7d6b0fa036"}},{"code_sha256_prefix":"7e484aba52303a9c","entry":"ax_decorate","repo":"ncar/credit-arxiv","repo_kind":"official","path":"libs/graph_utils.py","file_url":"https://github.com/ncar/credit-arxiv/blob/HEAD/libs/graph_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7e484aba52303a9c"}},{"code_sha256_prefix":"94b458853688a857","entry":"bootstrap_confidence_intervals","repo":"ncar/credit-arxiv","repo_kind":"official","path":"libs/score_utils.py","file_url":"https://github.com/ncar/credit-arxiv/blob/HEAD/libs/score_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"94b458853688a857"}},{"code_sha256_prefix":"f9087d3de0a605b0","entry":"bootstrap_core","repo":"ncar/credit-arxiv","repo_kind":"official","path":"libs/score_utils.py","file_url":"https://github.com/ncar/credit-arxiv/blob/HEAD/libs/score_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f9087d3de0a605b0"}},{"code_sha256_prefix":"cf3457aa78b93eed","entry":"calculate_ensemble_metrics","repo":"ncar/miles-credit","repo_kind":"official","path":"credit/applications/rollout_metrics_noisy_model.py","file_url":"https://github.com/ncar/miles-credit/blob/HEAD/credit/applications/rollout_metrics_noisy_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cf3457aa78b93eed"}},{"code_sha256_prefix":"dca6ac38782407c1","entry":"compute_metrics","repo":"ncar/miles-credit","repo_kind":"official","path":"credit/applications/rollout_metrics_noisy_model.py","file_url":"https://github.com/ncar/miles-credit/blob/HEAD/credit/applications/rollout_metrics_noisy_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dca6ac38782407c1"}},{"code_sha256_prefix":"5bd4ee70ce28a937","entry":"compute_spread_skill_metric","repo":"ncar/miles-credit","repo_kind":"official","path":"credit/applications/rollout_metrics_noisy_model.py","file_url":"https://github.com/ncar/miles-credit/blob/HEAD/credit/applications/rollout_metrics_noisy_model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5bd4ee70ce28a937"}},{"code_sha256_prefix":"710b7960d224b37c","entry":"conda_prefix_expr","repo":"ncar/miles-credit","repo_kind":"official","path":"credit/conda_env.py","file_url":"https://github.com/ncar/miles-credit/blob/HEAD/credit/conda_env.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"710b7960d224b37c"}},{"code_sha256_prefix":"f777c1387fe94d25","entry":"device_compatible_to","repo":"ncar/miles-credit","repo_kind":"official","path":"credit/data.py","file_url":"https://github.com/ncar/miles-credit/blob/HEAD/credit/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f777c1387fe94d25"}},{"code_sha256_prefix":"09c8479d9a5b3e06","entry":"extract","repo":"ncar/miles-credit","repo_kind":"official","path":"credit/diffusion.py","file_url":"https://github.com/ncar/miles-credit/blob/HEAD/credit/diffusion.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"09c8479d9a5b3e06"}},{"code_sha256_prefix":"b3aec1aecbdace4e","entry":"generate_datetime","repo":"ncar/miles-credit","repo_kind":"official","path":"credit/data.py","file_url":"https://github.com/ncar/miles-credit/blob/HEAD/credit/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b3aec1aecbdace4e"}},{"code_sha256_prefix":"5300cf7c2447c51c","entry":"k_to_c","repo":"ncar/miles-credit","repo_kind":"official","path":"credit/animation.py","file_url":"https://github.com/ncar/miles-credit/blob/HEAD/credit/animation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5300cf7c2447c51c"}},{"code_sha256_prefix":"ae167ca711bf0664","entry":"k_to_f","repo":"ncar/miles-credit","repo_kind":"official","path":"credit/animation.py","file_url":"https://github.com/ncar/miles-credit/blob/HEAD/credit/animation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ae167ca711bf0664"}},{"code_sha256_prefix":"6392ecdf0891587d","entry":"kgkg_to_gkg","repo":"ncar/miles-credit","repo_kind":"official","path":"credit/animation.py","file_url":"https://github.com/ncar/miles-credit/blob/HEAD/credit/animation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6392ecdf0891587d"}},{"code_sha256_prefix":"d9c0c00237e322bf","entry":"lead_to_index","repo":"ncar/credit-arxiv","repo_kind":"official","path":"libs/verif_utils.py","file_url":"https://github.com/ncar/credit-arxiv/blob/HEAD/libs/verif_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d9c0c00237e322bf"}},{"code_sha256_prefix":"f5811388d9ec253f","entry":"lg_box","repo":"ncar/credit-arxiv","repo_kind":"official","path":"libs/graph_utils.py","file_url":"https://github.com/ncar/credit-arxiv/blob/HEAD/libs/graph_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f5811388d9ec253f"}},{"code_sha256_prefix":"abcd739ba6239f32","entry":"log_uniform_noise","repo":"ncar/miles-credit","repo_kind":"official","path":"credit/diffusion.py","file_url":"https://github.com/ncar/miles-credit/blob/HEAD/credit/diffusion.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"abcd739ba6239f32"}},{"code_sha256_prefix":"92dcf06df9a09396","entry":"standard_normal_noise","repo":"ncar/miles-credit","repo_kind":"official","path":"credit/diffusion.py","file_url":"https://github.com/ncar/miles-credit/blob/HEAD/credit/diffusion.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"92dcf06df9a09396"}},{"code_sha256_prefix":"765c066faa550266","entry":"string_partial_format","repo":"ncar/credit-arxiv","repo_kind":"official","path":"libs/graph_utils.py","file_url":"https://github.com/ncar/credit-arxiv/blob/HEAD/libs/graph_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"765c066faa550266"}},{"code_sha256_prefix":"68b5e94ece062141","entry":"torchrun_path","repo":"ncar/miles-credit","repo_kind":"official","path":"credit/conda_env.py","file_url":"https://github.com/ncar/miles-credit/blob/HEAD/credit/conda_env.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"68b5e94ece062141"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}