{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/earthpt-a-foundation-model-for-earth","title":"EarthPT: a time series foundation model for Earth Observation","arxiv_id":"2309.07207","date":"2023-09-13","proceeding":null,"authors":["Michael J. Smith","Luke Fleming","James E. Geach"],"abstract":"We introduce EarthPT -- an Earth Observation (EO) pretrained transformer. EarthPT is a 700 million parameter decoding transformer foundation model trained in an autoregressive self-supervised manner and developed specifically with EO use-cases in mind. We demonstrate that EarthPT is an effective forecaster that can accurately predict future pixel-level surface reflectances across the 400-2300 nm range well into the future. For example, forecasts of the evolution of the Normalised Difference Vegetation Index (NDVI) have a typical error of approximately 0.05 (over a natural range of -1 -> 1) at the pixel level over a five month test set horizon, out-performing simple phase-folded models based on historical averaging. We also demonstrate that embeddings learnt by EarthPT hold semantically meaningful information and could be exploited for downstream tasks such as highly granular, dynamic land use classification. Excitingly, we note that the abundance of EO data provides us with -- in theory -- quadrillions of training tokens. Therefore, if we assume that EarthPT follows neural scaling laws akin to those derived for Large Language Models (LLMs), there is currently no data-imposed limit to scaling EarthPT and other similar `Large Observation Models.'","url_abs":"https://arxiv.org/abs/2309.07207v2","url_pdf":"https://arxiv.org/pdf/2309.07207v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"earthpt-a-foundation-model-for-earth","repo_url":"https://github.com/aspiaspace/earthpt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"earth-observation","task_name":"Earth Observation"},{"task_slug":"time-series-1","task_name":"Time Series"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2309.07207","atlas_url":"https://app.syntology.ai/?focus=2309.07207","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.07207"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/aspiaspace/earthpt","reach":null}],"summary":{"ran":2,"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b7589e0b66c2f55b","entry":"Block","repo":"aspiaspace/earthpt","repo_kind":"official","path":"src/model.py","file_url":"https://github.com/aspiaspace/earthpt/blob/HEAD/src/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b7589e0b66c2f55b"}},{"code_sha256_prefix":"2d8b3906adb95134","entry":"MLP","repo":"aspiaspace/earthpt","repo_kind":"official","path":"src/model.py","file_url":"https://github.com/aspiaspace/earthpt/blob/HEAD/src/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2d8b3906adb95134"}},{"code_sha256_prefix":"b174524f6040ef73","entry":"new_gelu","repo":"aspiaspace/earthPT","repo_kind":"official","path":"src/model.py","file_url":"https://github.com/aspiaspace/earthPT/blob/HEAD/src/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b174524f6040ef73"}},{"code_sha256_prefix":"92101b06ecbf1f1e","entry":"new_gelu","repo":"aspiaspace/EarthPT","repo_kind":"official","path":"src/model.py","file_url":"https://github.com/aspiaspace/EarthPT/blob/HEAD/src/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"92101b06ecbf1f1e"}},{"code_sha256_prefix":"94d98b2df1dae87f","entry":"GPT","repo":"aspiaspace/earthpt","repo_kind":"official","path":"src/model.py","file_url":"https://github.com/aspiaspace/earthpt/blob/HEAD/src/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"94d98b2df1dae87f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}