{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/exploring-complementary-strengths-of","title":"Exploring Complementary Strengths of Invariant and Equivariant Representations for Few-Shot Learning","arxiv_id":"2103.01315","date":"2021-03-01","proceeding":"CVPR 2021 1","authors":["Mamshad Nayeem Rizve","Salman Khan","Fahad Shahbaz Khan","Mubarak Shah"],"abstract":"In many real-world problems, collecting a large number of labeled samples is infeasible. Few-shot learning (FSL) is the dominant approach to address this issue, where the objective is to quickly adapt to novel categories in presence of a limited number of samples. FSL tasks have been predominantly solved by leveraging the ideas from gradient-based meta-learning and metric learning approaches. However, recent works have demonstrated the significance of powerful feature representations with a simple embedding network that can outperform existing sophisticated FSL algorithms. In this work, we build on this insight and propose a novel training mechanism that simultaneously enforces equivariance and invariance to a general set of geometric transformations. Equivariance or invariance has been employed standalone in the previous works; however, to the best of our knowledge, they have not been used jointly. Simultaneous optimization for both of these contrasting objectives allows the model to jointly learn features that are not only independent of the input transformation but also the features that encode the structure of geometric transformations. These complementary sets of features help generalize well to novel classes with only a few data samples. We achieve additional improvements by incorporating a novel self-supervised distillation objective. Our extensive experimentation shows that even without knowledge distillation our proposed method can outperform current state-of-the-art FSL methods on five popular benchmark datasets.","url_abs":"https://arxiv.org/abs/2103.01315v2","url_pdf":"https://arxiv.org/pdf/2103.01315v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"exploring-complementary-strengths-of","repo_url":"https://github.com/nayeemrizve/invariance-equivariance","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"few-shot-image-classification","task_name":"Few-Shot Image Classification"},{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"},{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"meta-learning","task_name":"Meta-Learning"},{"task_slug":"metric-learning","task_name":"Metric Learning"}],"methods":[{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/few-shot-image-classification-on-cifar-fs-5","task":"Few-Shot Image Classification","dataset":"CIFAR-FS 5-way (1-shot)","model":"Invariance-Equivariance","rank_in_archive_order":16,"of":38,"metrics":{"Accuracy":"77.87"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-image-classification-on-cifar-fs-5-1","task":"Few-Shot Image Classification","dataset":"CIFAR-FS 5-way (5-shot)","model":"Invariance-Equivariance","rank_in_archive_order":13,"of":39,"metrics":{"Accuracy":"89.74"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-image-classification-on-fc100-5-way","task":"Few-Shot Image Classification","dataset":"FC100 5-way (1-shot)","model":"Invariance-Equivariance","rank_in_archive_order":10,"of":22,"metrics":{"Accuracy":"47.76"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-image-classification-on-fc100-5-way-1","task":"Few-Shot Image Classification","dataset":"FC100 5-way (5-shot)","model":"Invariance-Equivariance","rank_in_archive_order":9,"of":22,"metrics":{"Accuracy":"65.3"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-image-classification-on-meta-dataset","task":"Few-Shot Image Classification","dataset":"Meta-Dataset","model":"Invariance-Equivariance","rank_in_archive_order":14,"of":22,"metrics":{"Accuracy":"68.89"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-image-classification-on-mini-2","task":"Few-Shot Image Classification","dataset":"Mini-Imagenet 5-way (1-shot)","model":"Invariance-Equivariance","rank_in_archive_order":46,"of":105,"metrics":{"Accuracy":"67.28"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-image-classification-on-mini-3","task":"Few-Shot Image Classification","dataset":"Mini-Imagenet 5-way (5-shot)","model":"Invariance-Equivariance","rank_in_archive_order":24,"of":95,"metrics":{"Accuracy":"84.78"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-image-classification-on-tiered","task":"Few-Shot Image Classification","dataset":"Tiered ImageNet 5-way (1-shot)","model":"Invariance-Equivariance","rank_in_archive_order":26,"of":49,"metrics":{"Accuracy":"72.21"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-image-classification-on-tiered-1","task":"Few-Shot Image Classification","dataset":"Tiered ImageNet 5-way (5-shot)","model":"Invariance-Equivariance","rank_in_archive_order":21,"of":51,"metrics":{"Accuracy":"87.08"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2103.01315","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2103.01315"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nayeemrizve/invariance-equivariance","reach":null}],"summary":{"ran":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5dc1dd6076a3cd95","entry":"ResNet","repo":"nayeemrizve/invariance-equivariance","repo_kind":"official","path":"models/resnet_inv_eq.py","file_url":"https://github.com/nayeemrizve/invariance-equivariance/blob/HEAD/models/resnet_inv_eq.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5dc1dd6076a3cd95"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}