{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-faster-training-of-global-covariance","title":"Towards Faster Training of Global Covariance Pooling Networks by Iterative Matrix Square Root Normalization","arxiv_id":"1712.01034","date":"2017-12-04","proceeding":"CVPR 2018 6","authors":["Peihua Li","Jiangtao Xie","Qilong Wang","Zilin Gao"],"abstract":"Global covariance pooling in convolutional neural networks has achieved\nimpressive improvement over the classical first-order pooling. Recent works\nhave shown matrix square root normalization plays a central role in achieving\nstate-of-the-art performance. However, existing methods depend heavily on\neigendecomposition (EIG) or singular value decomposition (SVD), suffering from\ninefficient training due to limited support of EIG and SVD on GPU. Towards\naddressing this problem, we propose an iterative matrix square root\nnormalization method for fast end-to-end training of global covariance pooling\nnetworks. At the core of our method is a meta-layer designed with loop-embedded\ndirected graph structure. The meta-layer consists of three consecutive\nnonlinear structured layers, which perform pre-normalization, coupled matrix\niteration and post-compensation, respectively. Our method is much faster than\nEIG or SVD based ones, since it involves only matrix multiplications, suitable\nfor parallel implementation on GPU. Moreover, the proposed network with ResNet\narchitecture can converge in much less epochs, further accelerating network\ntraining. On large-scale ImageNet, we achieve competitive performance superior\nto existing counterparts. By finetuning our models pre-trained on ImageNet, we\nestablish state-of-the-art results on three challenging fine-grained\nbenchmarks. The source code and network models will be available at\nhttp://www.peihuali.org/iSQRT-COV","url_abs":"http://arxiv.org/abs/1712.01034v2","url_pdf":"http://arxiv.org/pdf/1712.01034v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-faster-training-of-global-covariance","repo_url":"https://github.com/jiangtaoxie/fast-MPN-COV","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null},{"paper_slug":"towards-faster-training-of-global-covariance","repo_url":"https://github.com/OrkhanHI/Grab-AI-Computer-Vision-Challenge","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"towards-faster-training-of-global-covariance","repo_url":"https://github.com/jianshen92/stanford-car-grab-challenge","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"towards-faster-training-of-global-covariance","repo_url":"https://github.com/osmr/imgclsmob","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"mxnet","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"fine-grained-image-classification","task_name":"Fine-Grained Image Classification"},{"task_slug":"fine-grained-image-recognition","task_name":"Fine-Grained Image Recognition"},{"task_slug":null,"task_name":"GPU"},{"task_slug":"image-classification","task_name":"Image Classification"}],"methods":[{"method_slug":"mpn","method_name":"MPN"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/fine-grained-image-classification-on-cub-200-1","task":"Fine-Grained Image Classification","dataset":"CUB-200-2011","model":"MPN-COV","rank_in_archive_order":18,"of":30,"metrics":{"Accuracy":"88.7"},"uses_additional_data":false},{"leaderboard":"/sota/fine-grained-image-classification-on-fgvc","task":"Fine-Grained Image Classification","dataset":"FGVC Aircraft","model":"MPN-COV","rank_in_archive_order":45,"of":57,"metrics":{"Accuracy":"91.4%"},"uses_additional_data":false},{"leaderboard":"/sota/fine-grained-image-classification-on-stanford","task":"Fine-Grained Image Classification","dataset":"Stanford Cars","model":"MPN-COV","rank_in_archive_order":64,"of":83,"metrics":{"Accuracy":"93.3%"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1712.01034","atlas_url":"https://app.syntology.ai/?focus=1712.01034","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1712.01034"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jianshen92/stanford-car-grab-challenge","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jiangtaoxie/fast-MPN-COV","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/osmr/imgclsmob","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/OrkhanHI/Grab-AI-Computer-Vision-Challenge","reach":{"status":"unanswered"}}],"summary":{"ran_fixture":2,"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"3e3d33392b6ecdd9","entry":"CovpoolLayer","repo":"jiangtaoxie/fast-MPN-COV","repo_kind":"official","path":"src/representation/MPNCOV.py","file_url":"https://github.com/jiangtaoxie/fast-MPN-COV/blob/HEAD/src/representation/MPNCOV.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3e3d33392b6ecdd9"}},{"code_sha256_prefix":"3c7b6412a1bc9cf4","entry":"SqrtmLayer","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"3c7b6412a1bc9cf4"}},{"code_sha256_prefix":"5cee54d3483cad8f","entry":"TriuvecLayer","repo":"jiangtaoxie/fast-MPN-COV","repo_kind":"official","path":"src/representation/MPNCOV.py","file_url":"https://github.com/jiangtaoxie/fast-MPN-COV/blob/HEAD/src/representation/MPNCOV.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5cee54d3483cad8f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}