{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/cost-aggregation-is-all-you-need-for-few-shot","title":"Cost Aggregation Is All You Need for Few-Shot Segmentation","arxiv_id":"2112.11685","date":"2021-12-22","proceeding":null,"authors":["Sunghwan Hong","Seokju Cho","Jisu Nam","Seungryong Kim"],"abstract":"We introduce a novel cost aggregation network, dubbed Volumetric Aggregation with Transformers (VAT), to tackle the few-shot segmentation task by using both convolutions and transformers to efficiently handle high dimensional correlation maps between query and support. In specific, we propose our encoder consisting of volume embedding module to not only transform the correlation maps into more tractable size but also inject some convolutional inductive bias and volumetric transformer module for the cost aggregation. Our encoder has a pyramidal structure to let the coarser level aggregation to guide the finer level and enforce to learn complementary matching scores. We then feed the output into our affinity-aware decoder along with the projected feature maps for guiding the segmentation process. Combining these components, we conduct experiments to demonstrate the effectiveness of the proposed method, and our method sets a new state-of-the-art for all the standard benchmarks in few-shot segmentation task. Furthermore, we find that the proposed method attains state-of-the-art performance even for the standard benchmarks in semantic correspondence task although not specifically designed for this task. We also provide an extensive ablation study to validate our architectural choices. The trained weights and codes are available at: https://seokju-cho.github.io/VAT/.","url_abs":"https://arxiv.org/abs/2112.11685v1","url_pdf":"https://arxiv.org/pdf/2112.11685v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"cost-aggregation-is-all-you-need-for-few-shot","repo_url":"https://github.com/Seokju-Cho/Volumetric-Aggregation-Transformer","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"cost-aggregation-is-all-you-need-for-few-shot","repo_url":"https://github.com/SunghwanHong/Cost-Aggregation-transformers","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"GPL-3.0"}}],"tasks":[{"task_slug":"all","task_name":"All"},{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"few-shot-image-segmentation","task_name":"Few-Shot Semantic Segmentation"},{"task_slug":"inductive-bias","task_name":"Inductive Bias"},{"task_slug":"segmentation","task_name":"Segmentation"},{"task_slug":"semantic-correspondence","task_name":"Semantic correspondence"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/few-shot-semantic-segmentation-on-coco-20i-1","task":"Few-Shot Semantic Segmentation","dataset":"COCO-20i (1-shot)","model":"VAT (ResNet-50)","rank_in_archive_order":59,"of":85,"metrics":{"Mean IoU":"41.3"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-semantic-segmentation-on-coco-20i-5","task":"Few-Shot Semantic Segmentation","dataset":"COCO-20i (5-shot)","model":"VAT (ResNet-50)","rank_in_archive_order":52,"of":81,"metrics":{"Mean IoU":"47.9"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-semantic-segmentation-on-fss-1000-1","task":"Few-Shot Semantic Segmentation","dataset":"FSS-1000 (1-shot)","model":"VAT","rank_in_archive_order":8,"of":24,"metrics":{"Mean IoU":"90.0"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-semantic-segmentation-on-fss-1000-5","task":"Few-Shot Semantic Segmentation","dataset":"FSS-1000 (5-shot)","model":"VAT","rank_in_archive_order":5,"of":22,"metrics":{"Mean IoU":"90.6"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-semantic-segmentation-on-pascal-5i-1","task":"Few-Shot Semantic Segmentation","dataset":"PASCAL-5i (1-Shot)","model":"VAT","rank_in_archive_order":26,"of":105,"metrics":{"Mean IoU":"67.5"},"uses_additional_data":false},{"leaderboard":"/sota/few-shot-semantic-segmentation-on-pascal-5i-5","task":"Few-Shot Semantic Segmentation","dataset":"PASCAL-5i (5-Shot)","model":"VAT","rank_in_archive_order":24,"of":96,"metrics":{"Mean IoU":"71.6"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-correspondence-on-pf-pascal","task":"Semantic correspondence","dataset":"PF-PASCAL","model":"VAT","rank_in_archive_order":7,"of":15,"metrics":{"PCK":"92.3"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-correspondence-on-pf-willow","task":"Semantic correspondence","dataset":"PF-WILLOW","model":"VAT","rank_in_archive_order":3,"of":8,"metrics":{"PCK":"81.0"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-correspondence-on-spair-71k","task":"Semantic correspondence","dataset":"SPair-71k","model":"VAT","rank_in_archive_order":13,"of":22,"metrics":{"PCK":"54.2"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2112.11685","atlas_url":"https://app.syntology.ai/?focus=2112.11685","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}