{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/vision-foundation-models-can-they-be-applied","title":"Bridging the Gap: Examining Vision Foundation Models for Optical and Radio Astronomy Applications","arxiv_id":"2409.11175","date":"2024-09-17","proceeding":null,"authors":["E. Lastufka","O. Bait","M. Drozdova","V. Kinakh","D. Piras","M. Audard","M. Dessauges-Zavadsky","T. Holotyak","D. Schaerer","S. Voloshynovskiy"],"abstract":"Vision foundation models, which have demonstrated significant potential in many multimedia applications, are often underutilized in the natural sciences. This is primarily due to mismatches between the nature of domain-specific scientific data and the typical training data used for foundation models, leading to distribution shifts. Scientific data often differ substantially in structure and characteristics, and researchers frequently face the challenge of optimizing model performance with limited labeled data of only a few hundred or thousand images. This work evaluates the performance of vision foundation models in astrophysics, with a focus on identifying the best practices for adapting them to domain-specific datasets. We aim to establish a framework for selecting, fine-tuning, and optimizing these models for common tasks in optical and radio astronomy. We compared multiple foundation models, including self-supervised, weakly supervised, and distillation-based architectures, across two representative optical and radio datasets. Experiments involved different fine-tuning strategies, projector heads, and data preprocessing techniques, with performance evaluated on classification and detection metrics. Features extracted by specific foundation models improved classification accuracy for optical galaxy images compared to conventional supervised training. Similarly, these models achieved equivalent or superior performance in object detection tasks with radio images. However, classification performance for radio galaxy images was generally poor, often falling short of supervised approaches. These findings demonstrate that vision foundation models can be effectively adapted to astrophysical applications, provided practitioners iterate on model selection, training strategies, and data handling.","url_abs":"https://arxiv.org/abs/2409.11175v4","url_pdf":"https://arxiv.org/pdf/2409.11175v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"vision-foundation-models-can-they-be-applied","repo_url":"https://github.com/elastufka/fm4astro","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"GPL-3.0"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2409.11175","atlas_url":"https://app.syntology.ai/?focus=2409.11175","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2409.11175"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/elastufka/fm4astro","reach":{"status":"ok","spdx":"GPL-3.0"}}],"summary":{"ran":13,"unverified":3},"by_repo_kind":{"official":{"samples":16,"ran":13,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":16,"samples":[{"code_sha256_prefix":"1344900d0c914026","entry":"collate_fn","repo":"elastufka/fm4astro","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/elastufka/fm4astro/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"1344900d0c914026"}},{"code_sha256_prefix":"f92f16a13cfc2e6a","entry":"default_config","repo":"elastufka/fm4astro","repo_kind":"official","path":"evaluator.py","file_url":"https://github.com/elastufka/fm4astro/blob/HEAD/evaluator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"f92f16a13cfc2e6a"}},{"code_sha256_prefix":"3a7b80e5682f8cd3","entry":"eval_binary","repo":"elastufka/fm4astro","repo_kind":"official","path":"classifier_model.py","file_url":"https://github.com/elastufka/fm4astro/blob/HEAD/classifier_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"3a7b80e5682f8cd3"}},{"code_sha256_prefix":"ba8c7028bbbe7a67","entry":"fit_pca","repo":"elastufka/fm4astro","repo_kind":"official","path":"evaluator.py","file_url":"https://github.com/elastufka/fm4astro/blob/HEAD/evaluator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"ba8c7028bbbe7a67"}},{"code_sha256_prefix":"af9597222f43fdd4","entry":"freeze_resnet_layers","repo":"elastufka/fm4astro","repo_kind":"official","path":"train_frcnn.py","file_url":"https://github.com/elastufka/fm4astro/blob/HEAD/train_frcnn.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"af9597222f43fdd4"}},{"code_sha256_prefix":"662e84e65de78e84","entry":"get_all_configs","repo":"elastufka/fm4astro","repo_kind":"official","path":"downstream.py","file_url":"https://github.com/elastufka/fm4astro/blob/HEAD/downstream.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"662e84e65de78e84"}},{"code_sha256_prefix":"a4eaddefa2df529e","entry":"get_last_layer","repo":"elastufka/fm4astro","repo_kind":"official","path":"classifier_model.py","file_url":"https://github.com/elastufka/fm4astro/blob/HEAD/classifier_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"a4eaddefa2df529e"}},{"code_sha256_prefix":"038de18b9aae0811","entry":"jpg_loader","repo":"elastufka/fm4astro","repo_kind":"official","path":"fm_datasets.py","file_url":"https://github.com/elastufka/fm4astro/blob/HEAD/fm_datasets.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"038de18b9aae0811"}},{"code_sha256_prefix":"1291d15dae744bb4","entry":"norm_label","repo":"elastufka/fm4astro","repo_kind":"official","path":"evaluator.py","file_url":"https://github.com/elastufka/fm4astro/blob/HEAD/evaluator.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"1291d15dae744bb4"}},{"code_sha256_prefix":"797d7b61a53aa714","entry":"npy_loader","repo":"elastufka/fm4astro","repo_kind":"official","path":"fm_datasets.py","file_url":"https://github.com/elastufka/fm4astro/blob/HEAD/fm_datasets.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"797d7b61a53aa714"}},{"code_sha256_prefix":"d0fdd2f3703ea795","entry":"resize_boxes","repo":"elastufka/fm4astro","repo_kind":"official","path":"myGRCNNtransform.py","file_url":"https://github.com/elastufka/fm4astro/blob/HEAD/myGRCNNtransform.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"d0fdd2f3703ea795"}},{"code_sha256_prefix":"76024c7f5ea7c958","entry":"resize_keypoints","repo":"elastufka/fm4astro","repo_kind":"official","path":"myGRCNNtransform.py","file_url":"https://github.com/elastufka/fm4astro/blob/HEAD/myGRCNNtransform.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"76024c7f5ea7c958"}},{"code_sha256_prefix":"4709fd173194e4ad","entry":"train_and_eval_model","repo":"elastufka/fm4astro","repo_kind":"official","path":"classifier_model.py","file_url":"https://github.com/elastufka/fm4astro/blob/HEAD/classifier_model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"4709fd173194e4ad"}},{"code_sha256_prefix":"0d4c7f84fc4fb4ad","entry":"create_model","repo":"elastufka/fm4astro","repo_kind":"official","path":"fasterrcnn_resnet18.py","file_url":"https://github.com/elastufka/fm4astro/blob/HEAD/fasterrcnn_resnet18.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"0d4c7f84fc4fb4ad"}},{"code_sha256_prefix":"8ad5e131c9f9151f","entry":"freeze_ViT_blocks","repo":"elastufka/fm4astro","repo_kind":"official","path":"train_frcnn.py","file_url":"https://github.com/elastufka/fm4astro/blob/HEAD/train_frcnn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"8ad5e131c9f9151f"}},{"code_sha256_prefix":"5549b6c9281cfcfe","entry":"transform","repo":"elastufka/fm4astro","repo_kind":"official","path":"finetune.py","file_url":"https://github.com/elastufka/fm4astro/blob/HEAD/finetune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"5549b6c9281cfcfe"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}