{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/seasonal-contrast-unsupervised-pre-training","title":"Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data","arxiv_id":"2103.16607","date":"2021-03-30","proceeding":"ICCV 2021 10","authors":["Oscar Mañas","Alexandre Lacoste","Xavier Giro-i-Nieto","David Vazquez","Pau Rodriguez"],"abstract":"Remote sensing and automatic earth monitoring are key to solve global-scale challenges such as disaster prevention, land use monitoring, or tackling climate change. Although there exist vast amounts of remote sensing data, most of it remains unlabeled and thus inaccessible for supervised learning algorithms. Transfer learning approaches can reduce the data requirements of deep learning algorithms. However, most of these methods are pre-trained on ImageNet and their generalization to remote sensing imagery is not guaranteed due to the domain gap. In this work, we propose Seasonal Contrast (SeCo), an effective pipeline to leverage unlabeled data for in-domain pre-training of remote sensing representations. The SeCo pipeline is composed of two parts. First, a principled procedure to gather large-scale, unlabeled and uncurated remote sensing datasets containing images from multiple Earth locations at different timestamps. Second, a self-supervised algorithm that takes advantage of time and position invariance to learn transferable representations for remote sensing applications. We empirically show that models trained with SeCo achieve better performance than their ImageNet pre-trained counterparts and state-of-the-art self-supervised learning methods on multiple downstream tasks. The datasets and models in SeCo will be made public to facilitate transfer learning and enable rapid progress in remote sensing applications.","url_abs":"https://arxiv.org/abs/2103.16607v2","url_pdf":"https://arxiv.org/pdf/2103.16607v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"seasonal-contrast-unsupervised-pre-training","repo_url":"https://github.com/ElementAI/seasonal-contrast","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"seasonal-contrast-unsupervised-pre-training","repo_url":"https://github.com/floatingstarZ/GeRSP","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"seasonal-contrast-unsupervised-pre-training","repo_url":"https://github.com/floatingstarz/mutdet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"seasonal-contrast-unsupervised-pre-training","repo_url":"https://github.com/hewanshrestha/why-self-supervision-in-time","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"seasonal-contrast-unsupervised-pre-training","repo_url":"https://github.com/servicenow/seasonal-contrast","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"change-detection","task_name":"Change Detection"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"},{"task_slug":"unsupervised-pre-training","task_name":"Unsupervised Pre-training"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/change-detection-on-gvlm","task":"Change Detection","dataset":"GVLM","model":"SeCo","rank_in_archive_order":2,"of":4,"metrics":{"F1":"90.4"},"uses_additional_data":false},{"leaderboard":"/sota/change-detection-on-oscd-13ch","task":"Change Detection","dataset":"OSCD - 13ch","model":"SeCo","rank_in_archive_order":4,"of":6,"metrics":{"F1":"46.94","Precision":"38.06"},"uses_additional_data":true}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2103.16607","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2103.16607"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/floatingstarZ/GeRSP","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/floatingstarz/mutdet","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hewanshrestha/why-self-supervision-in-time","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/servicenow/seasonal-contrast","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/ElementAI/seasonal-contrast","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":8},"by_repo_kind":{"official":{"samples":6,"ran":0,"repositories":1},"listed":{"samples":2,"ran":0,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"35c74641cae2d081","entry":"batch_shuffle_ddp","repo":"servicenow/seasonal-contrast","repo_kind":"listed","path":"models/moco2_module.py","file_url":"https://github.com/servicenow/seasonal-contrast/blob/HEAD/models/moco2_module.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"35c74641cae2d081"}},{"code_sha256_prefix":"84c3492f31564f3e","entry":"batch_unshuffle_ddp","repo":"ElementAI/seasonal-contrast","repo_kind":"official","path":"models/moco2_module.py","file_url":"https://github.com/ElementAI/seasonal-contrast/blob/HEAD/models/moco2_module.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"84c3492f31564f3e"}},{"code_sha256_prefix":"2ea140a31f1421c4","entry":"calculate_accuracy","repo":"floatingstarZ/GeRSP","repo_kind":"listed","path":"mmselfsup/models/algorithms/moco_gersp.py","file_url":"https://github.com/floatingstarZ/GeRSP/blob/HEAD/mmselfsup/models/algorithms/moco_gersp.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2ea140a31f1421c4"}},{"code_sha256_prefix":"73cecca9f3575f09","entry":"concat_all_gather","repo":"ElementAI/seasonal-contrast","repo_kind":"official","path":"models/moco2_module.py","file_url":"https://github.com/ElementAI/seasonal-contrast/blob/HEAD/models/moco2_module.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"73cecca9f3575f09"}},{"code_sha256_prefix":"8cea7a513c856e17","entry":"get_experiment_name","repo":"ElementAI/seasonal-contrast","repo_kind":"official","path":"main_bigearthnet.py","file_url":"https://github.com/ElementAI/seasonal-contrast/blob/HEAD/main_bigearthnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8cea7a513c856e17"}},{"code_sha256_prefix":"da02260fb5561b46","entry":"get_experiment_name","repo":"ElementAI/seasonal-contrast","repo_kind":"official","path":"main_pretrain.py","file_url":"https://github.com/ElementAI/seasonal-contrast/blob/HEAD/main_pretrain.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"da02260fb5561b46"}},{"code_sha256_prefix":"47e0aa4f82fd7af1","entry":"get_segmentation_model","repo":"ElementAI/seasonal-contrast","repo_kind":"official","path":"models/segmentation.py","file_url":"https://github.com/ElementAI/seasonal-contrast/blob/HEAD/models/segmentation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"47e0aa4f82fd7af1"}},{"code_sha256_prefix":"15c4c0f9b256fee4","entry":"normalize","repo":"ElementAI/seasonal-contrast","repo_kind":"official","path":"datasets/bigearthnet_dataset.py","file_url":"https://github.com/ElementAI/seasonal-contrast/blob/HEAD/datasets/bigearthnet_dataset.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"15c4c0f9b256fee4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}