{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/connect-later-improving-fine-tuning-for","title":"Connect Later: Improving Fine-tuning for Robustness with Targeted Augmentations","arxiv_id":"2402.03325","date":"2024-01-08","proceeding":null,"authors":["Helen Qu","Sang Michael Xie"],"abstract":"Models trained on a labeled source domain (e.g., labeled images from wildlife camera traps) often generalize poorly when deployed on an out-of-distribution (OOD) target domain (e.g., images from new camera trap locations). In the domain adaptation setting where unlabeled target data is available, self-supervised pretraining (e.g., masked autoencoding or contrastive learning) is a promising method to mitigate this performance drop. Pretraining improves OOD error when the generic data augmentations used (e.g., masking or cropping) connect the source and target domains, which may be far apart in the input space. In this paper, we show on real-world tasks that standard fine-tuning after pretraining does not consistently improve OOD error over simply training from scratch on labeled source data. To better leverage pretraining for distribution shifts, we propose Connect Later: after pretraining with generic augmentations, fine-tune with targeted augmentations designed with knowledge of the distribution shift. Pretraining learns good representations within the source and target domains, while targeted augmentations connect the domains better during fine-tuning. Connect Later improves average OOD error over standard fine-tuning and supervised learning with targeted augmentations on 4 real-world datasets: Connect Later achieves the state-of-the-art on astronomical time-series classification (AstroClassification) by 2.5%, wildlife species identification (iWildCam-WILDS) with ResNet-50 by 0.9%, and tumor identification (Camelyon17-WILDS) with DenseNet121 by 1.1%; as well as best performance on a new dataset for astronomical time-series redshift prediction (Redshifts) by 0.03 RMSE (11% relative). Code and datasets are available at https://github.com/helenqu/connect-later.","url_abs":"https://arxiv.org/abs/2402.03325v2","url_pdf":"https://arxiv.org/pdf/2402.03325v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"connect-later-improving-fine-tuning-for","repo_url":"https://github.com/helenqu/connect-later","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"domain-adaptation","task_name":"Domain Adaptation"},{"task_slug":"time-series-1","task_name":"Time Series"},{"task_slug":"time-series-classification","task_name":"Time Series Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2402.03325","atlas_url":"https://app.syntology.ai/?focus=2402.03325","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.03325"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/helenqu/connect-later","reach":{"status":"ok"}}],"summary":{"ran":3,"unverified":3},"by_repo_kind":{"official":{"samples":6,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"3f40eee76eaed297","entry":"convert_to_pandas_period","repo":"helenqu/connect-later","repo_kind":"official","path":"connect_later/dataset_preprocess_raw.py","file_url":"https://github.com/helenqu/connect-later/blob/HEAD/connect_later/dataset_preprocess_raw.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3f40eee76eaed297"}},{"code_sha256_prefix":"447e52d440681920","entry":"prepare_model_input","repo":"helenqu/connect-later","repo_kind":"official","path":"connect_later/pretrain.py","file_url":"https://github.com/helenqu/connect-later/blob/HEAD/connect_later/pretrain.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"447e52d440681920"}},{"code_sha256_prefix":"d9beb42690d42263","entry":"setup_model_config","repo":"helenqu/connect-later","repo_kind":"official","path":"connect_later/pretrain.py","file_url":"https://github.com/helenqu/connect-later/blob/HEAD/connect_later/pretrain.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d9beb42690d42263"}},{"code_sha256_prefix":"8339f116bb89252e","entry":"get_dataset","repo":"helenqu/connect-later","repo_kind":"official","path":"connect_later/pretrain.py","file_url":"https://github.com/helenqu/connect-later/blob/HEAD/connect_later/pretrain.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8339f116bb89252e"}},{"code_sha256_prefix":"59c3d5e2ae6e0a8d","entry":"normalize_all","repo":"helenqu/connect-later","repo_kind":"official","path":"connect_later/dataset_preprocess_raw.py","file_url":"https://github.com/helenqu/connect-later/blob/HEAD/connect_later/dataset_preprocess_raw.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"59c3d5e2ae6e0a8d"}},{"code_sha256_prefix":"5f393d809299394f","entry":"normalize_by_channel","repo":"helenqu/connect-later","repo_kind":"official","path":"connect_later/dataset_preprocess_raw.py","file_url":"https://github.com/helenqu/connect-later/blob/HEAD/connect_later/dataset_preprocess_raw.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5f393d809299394f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}