{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/stochastic-subsampling-with-average-pooling","title":"Stochastic Subsampling With Average Pooling","arxiv_id":"2409.16630","date":"2024-09-25","proceeding":null,"authors":["Bum Jun Kim","Sang Woo Kim"],"abstract":"Regularization of deep neural networks has been an important issue to achieve higher generalization performance without overfitting problems. Although the popular method of Dropout provides a regularization effect, it causes inconsistent properties in the output, which may degrade the performance of deep neural networks. In this study, we propose a new module called stochastic average pooling, which incorporates Dropout-like stochasticity in pooling. We describe the properties of stochastic subsampling and average pooling and leverage them to design a module without any inconsistency problem. The stochastic average pooling achieves a regularization effect without any potential performance degradation due to the inconsistency issue and can easily be plugged into existing architectures of deep neural networks. Experiments demonstrate that replacing existing average pooling with stochastic average pooling yields consistent improvements across a variety of tasks, datasets, and models.","url_abs":"https://arxiv.org/abs/2409.16630v1","url_pdf":"https://arxiv.org/pdf/2409.16630v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"fine-grained-image-classification","task_name":"Fine-Grained Image Classification"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"dropout","method_name":"Dropout"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/fine-grained-image-classification-on-caltech","task":"Fine-Grained Image Classification","dataset":"Caltech-101","model":"SE-ResNet-101 (SAP)","rank_in_archive_order":14,"of":18,"metrics":{"Top-1 Error Rate":"15.949%"},"uses_additional_data":false},{"leaderboard":"/sota/fine-grained-image-classification-on-oxford-2","task":"Fine-Grained Image Classification","dataset":"Oxford-IIIT Pets","model":"SE-ResNet-101 (SAP)","rank_in_archive_order":16,"of":19,"metrics":{"Accuracy":"86.011"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-cifar-10","task":"Image Classification","dataset":"CIFAR-10","model":"ResNet-110 (SAP)","rank_in_archive_order":167,"of":265,"metrics":{"Percentage correct":"93.861"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-cifar-100","task":"Image Classification","dataset":"CIFAR-100","model":"ResNet-110 (SAP)","rank_in_archive_order":167,"of":211,"metrics":{"Percentage correct":"72.537"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-stanford-cars","task":"Image Classification","dataset":"Stanford Cars","model":"SE-ResNet-101 (SAP)","rank_in_archive_order":21,"of":24,"metrics":{"Accuracy":"85.812"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-coco-2017","task":"Object Detection","dataset":"COCO 2017","model":"DyHead (SAP)","rank_in_archive_order":6,"of":24,"metrics":{"AP":"42.1","AP50":"59.4","AP75":"45.9"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-isprs-potsdam","task":"Semantic Segmentation","dataset":"ISPRS Potsdam","model":"PSPNet (SAP)","rank_in_archive_order":17,"of":20,"metrics":{"Mean IoU":"74.3","Overall Accuracy":"88.56"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-isprs-vaihingen","task":"Semantic Segmentation","dataset":"ISPRS Vaihingen","model":"UPerNet (SAP)","rank_in_archive_order":10,"of":12,"metrics":{"Category mIoU":"73.27","Overall Accuracy":"90.14"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}