{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/rethinking-generalization-in-few-shot-1","title":"Rethinking Generalization in Few-Shot Classification","arxiv_id":"2206.07267","date":"2022-06-15","proceeding":null,"authors":["Markus Hiller","Rongkai Ma","Mehrtash Harandi","Tom Drummond"],"abstract":"Single image-level annotations only correctly describe an often small subset of an image's content, particularly when complex real-world scenes are depicted. While this might be acceptable in many classification scenarios, it poses a significant challenge for applications where the set of classes differs significantly between training and test time. In this paper, we take a closer look at the implications in the context of $\\textit{few-shot learning}$. Splitting the input samples into patches and encoding these via the help of Vision Transformers allows us to establish semantic correspondences between local regions across images and independent of their respective class. The most informative patch embeddings for the task at hand are then determined as a function of the support set via online optimization at inference time, additionally providing visual interpretability of `$\\textit{what matters most}$' in the image. We build on recent advances in unsupervised training of networks via masked image modelling to overcome the lack of fine-grained labels and learn the more general statistical structure of the data while avoiding negative image-level annotation influence, $\\textit{aka}$ supervision collapse. Experimental results show the competitiveness of our approach, achieving new state-of-the-art results on four popular few-shot classification benchmarks for $5$-shot and $1$-shot scenarios.","url_abs":"https://arxiv.org/abs/2206.07267v3","url_pdf":"https://arxiv.org/pdf/2206.07267v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"rethinking-generalization-in-few-shot-1","repo_url":"https://github.com/mrkshllr/FewTURE","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"few-shot-image-classification","task_name":"Few-Shot Image Classification"},{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"swin-transformer","method_name":"Swin Transformer"},{"method_slug":"test","method_name":"Test"},{"method_slug":"vision-transformer","method_name":"Vision Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/few-shot-image-classification-on-cifar-fs-5","task":"Few-Shot Image Classification","dataset":"CIFAR-FS 5-way (1-shot)","model":"FewTURE","rank_in_archive_order":17,"of":38,"metrics":{"Accuracy":"77.76"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-image-classification-on-cifar-fs-5-1","task":"Few-Shot Image Classification","dataset":"CIFAR-FS 5-way (5-shot)","model":"FewTURE","rank_in_archive_order":18,"of":39,"metrics":{"Accuracy":"88.90"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-image-classification-on-fc100-5-way","task":"Few-Shot Image Classification","dataset":"FC100 5-way (1-shot)","model":"FewTURE","rank_in_archive_order":11,"of":22,"metrics":{"Accuracy":"47.68"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-image-classification-on-fc100-5-way-1","task":"Few-Shot Image Classification","dataset":"FC100 5-way (5-shot)","model":"FewTURE","rank_in_archive_order":12,"of":22,"metrics":{"Accuracy":"63.81"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-image-classification-on-mini-2","task":"Few-Shot Image Classification","dataset":"Mini-Imagenet 5-way (1-shot)","model":"FewTURE","rank_in_archive_order":27,"of":105,"metrics":{"Accuracy":"72.40"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-image-classification-on-mini-3","task":"Few-Shot Image Classification","dataset":"Mini-Imagenet 5-way (5-shot)","model":"FewTURE","rank_in_archive_order":21,"of":95,"metrics":{"Accuracy":"86.38"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-image-classification-on-tiered","task":"Few-Shot Image Classification","dataset":"Tiered ImageNet 5-way (1-shot)","model":"FewTURE","rank_in_archive_order":18,"of":49,"metrics":{"Accuracy":"76.32"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-image-classification-on-tiered-1","task":"Few-Shot Image Classification","dataset":"Tiered ImageNet 5-way (5-shot)","model":"FewTURE","rank_in_archive_order":7,"of":51,"metrics":{"Accuracy":"89.96"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2206.07267","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.07267"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mrkshllr/FewTURE","reach":null}],"summary":{"ran_honours":1,"ran_draft_wrong":1,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"7322771f6296426b","entry":"compute_emb_cosine_similarity","repo":"mrkshllr/FewTURE","repo_kind":"official","path":"train_metatrain_FewTURE.py","file_url":"https://github.com/mrkshllr/FewTURE/blob/HEAD/train_metatrain_FewTURE.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"7322771f6296426b"}},{"code_sha256_prefix":"79257feee546698e","entry":"get_patch_embeddings","repo":"mrkshllr/FewTURE","repo_kind":"official","path":"train_metatrain_FewTURE.py","file_url":"https://github.com/mrkshllr/FewTURE/blob/HEAD/train_metatrain_FewTURE.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"79257feee546698e"}},{"code_sha256_prefix":"d19384c7d90e54d8","entry":"set_up_dataset","repo":"mrkshllr/FewTURE","repo_kind":"official","path":"train_metatrain_FewTURE.py","file_url":"https://github.com/mrkshllr/FewTURE/blob/HEAD/train_metatrain_FewTURE.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"d19384c7d90e54d8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}