{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-new-benchmark-for-evaluation-of-cross","title":"A Broader Study of Cross-Domain Few-Shot Learning","arxiv_id":"1912.07200","date":"2019-12-16","proceeding":"ECCV 2020 8","authors":["Yunhui Guo","Noel C. Codella","Leonid Karlinsky","James V. Codella","John R. Smith","Kate Saenko","Tajana Rosing","Rogerio Feris"],"abstract":"Recent progress on few-shot learning largely relies on annotated data for meta-learning: base classes sampled from the same domain as the novel classes. However, in many applications, collecting data for meta-learning is infeasible or impossible. This leads to the cross-domain few-shot learning problem, where there is a large shift between base and novel class domains. While investigations of the cross-domain few-shot scenario exist, these works are limited to natural images that still contain a high degree of visual similarity. No work yet exists that examines few-shot learning across different imaging methods seen in real world scenarios, such as aerial and medical imaging. In this paper, we propose the Broader Study of Cross-Domain Few-Shot Learning (BSCD-FSL) benchmark, consisting of image data from a diverse assortment of image acquisition methods. This includes natural images, such as crop disease images, but additionally those that present with an increasing dissimilarity to natural images, such as satellite images, dermatology images, and radiology images. Extensive experiments on the proposed benchmark are performed to evaluate state-of-art meta-learning approaches, transfer learning approaches, and newer methods for cross-domain few-shot learning. The results demonstrate that state-of-art meta-learning methods are surprisingly outperformed by earlier meta-learning approaches, and all meta-learning methods underperform in relation to simple fine-tuning by 12.8% average accuracy. Performance gains previously observed with methods specialized for cross-domain few-shot learning vanish in this more challenging benchmark. Finally, accuracy of all methods tend to correlate with dataset similarity to natural images, verifying the value of the benchmark to better represent the diversity of data seen in practice and guiding future research.","url_abs":"https://arxiv.org/abs/1912.07200v2","url_pdf":"https://arxiv.org/pdf/1912.07200v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-new-benchmark-for-evaluation-of-cross","repo_url":"https://github.com/IBM/cdfsl-benchmark","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"a-new-benchmark-for-evaluation-of-cross","repo_url":"https://github.com/hushell/pmf_cvpr22","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"cross-domain-few-shot","task_name":"Cross-Domain Few-Shot"},{"task_slug":"few-shot-image-classification","task_name":"Few-Shot Image Classification"},{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"},{"task_slug":"meta-learning","task_name":"Meta-Learning"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"},{"task_slug":"cross-domain-few-shot-learning","task_name":"cross-domain few-shot learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/cross-domain-few-shot-on-cub","task":"Cross-Domain Few-Shot","dataset":"CUB","model":"BSCD-FSL","rank_in_archive_order":9,"of":9,"metrics":{"5 shot":"64.14"},"uses_additional_data":false},{"leaderboard":"/sota/cross-domain-few-shot-on-chestx","task":"Cross-Domain Few-Shot","dataset":"ChestX","model":"BSCD-FSL","rank_in_archive_order":4,"of":11,"metrics":{"5 shot":"25.97"},"uses_additional_data":false},{"leaderboard":"/sota/cross-domain-few-shot-on-cropdisease","task":"Cross-Domain Few-Shot","dataset":"CropDisease","model":"BSCD-FSL","rank_in_archive_order":7,"of":9,"metrics":{"5 shot":"89.25"},"uses_additional_data":false},{"leaderboard":"/sota/cross-domain-few-shot-on-eurosat","task":"Cross-Domain Few-Shot","dataset":"EuroSAT","model":"BSCD-FSL","rank_in_archive_order":9,"of":11,"metrics":{"5 shot":"79.08"},"uses_additional_data":false},{"leaderboard":"/sota/cross-domain-few-shot-on-isic2018","task":"Cross-Domain Few-Shot","dataset":"ISIC2018","model":"BSCD-FSL","rank_in_archive_order":3,"of":11,"metrics":{"5 shot":"48.11"},"uses_additional_data":false},{"leaderboard":"/sota/cross-domain-few-shot-on-places","task":"Cross-Domain Few-Shot","dataset":"Places","model":"BSCD-FSL","rank_in_archive_order":8,"of":8,"metrics":{"5 shot":"70.06"},"uses_additional_data":false},{"leaderboard":"/sota/cross-domain-few-shot-on-plantae","task":"Cross-Domain Few-Shot","dataset":"Plantae","model":"BSCD-FSL","rank_in_archive_order":3,"of":8,"metrics":{"5 shot":"59.27"},"uses_additional_data":false},{"leaderboard":"/sota/cross-domain-few-shot-on-cars","task":"Cross-Domain Few-Shot","dataset":"cars","model":"BSCD-FSL","rank_in_archive_order":3,"of":8,"metrics":{"5 shot":"52.08"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1912.07200","atlas_url":"https://app.syntology.ai/?focus=1912.07200","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1912.07200"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hushell/pmf_cvpr22","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/IBM/cdfsl-benchmark","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":4,"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"listed":{"samples":7,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"592e3a34636ae458","entry":"ResNet10","repo":"IBM/cdfsl-benchmark","repo_kind":"listed","path":"backbone.py","file_url":"https://github.com/IBM/cdfsl-benchmark/blob/HEAD/backbone.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"592e3a34636ae458"}},{"code_sha256_prefix":"4dd319c45d372246","entry":"euclidean_dist","repo":"IBM/cdfsl-benchmark","repo_kind":"listed","path":"methods/protonet.py","file_url":"https://github.com/IBM/cdfsl-benchmark/blob/HEAD/methods/protonet.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4dd319c45d372246"}},{"code_sha256_prefix":"400a8db2fb9b1633","entry":"get_assigned_file","repo":"IBM/cdfsl-benchmark","repo_kind":"listed","path":"io_utils.py","file_url":"https://github.com/IBM/cdfsl-benchmark/blob/HEAD/io_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"400a8db2fb9b1633"}},{"code_sha256_prefix":"f1324589fc273593","entry":"one_hot","repo":"IBM/cdfsl-benchmark","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/IBM/cdfsl-benchmark/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f1324589fc273593"}},{"code_sha256_prefix":"de09514deda6f881","entry":"sparsity","repo":"IBM/cdfsl-benchmark","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/IBM/cdfsl-benchmark/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"de09514deda6f881"}},{"code_sha256_prefix":"4bd64c34ba45e089","entry":"get_resume_file","repo":"IBM/cdfsl-benchmark","repo_kind":"listed","path":"io_utils.py","file_url":"https://github.com/IBM/cdfsl-benchmark/blob/HEAD/io_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4bd64c34ba45e089"}},{"code_sha256_prefix":"15c45627ecb64c8d","entry":"parse_args","repo":"IBM/cdfsl-benchmark","repo_kind":"listed","path":"io_utils.py","file_url":"https://github.com/IBM/cdfsl-benchmark/blob/HEAD/io_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"15c45627ecb64c8d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}