{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ghostnetv3-exploring-the-training-strategies","title":"GhostNetV3: Exploring the Training Strategies for Compact Models","arxiv_id":"2404.11202","date":"2024-04-17","proceeding":null,"authors":["Zhenhua Liu","Zhiwei Hao","Kai Han","Yehui Tang","Yunhe Wang"],"abstract":"Compact neural networks are specially designed for applications on edge devices with faster inference speed yet modest performance. However, training strategies of compact models are borrowed from that of conventional models at present, which ignores their difference in model capacity and thus may impede the performance of compact models. In this paper, by systematically investigating the impact of different training ingredients, we introduce a strong training strategy for compact models. We find that the appropriate designs of re-parameterization and knowledge distillation are crucial for training high-performance compact models, while some commonly used data augmentations for training conventional models, such as Mixup and CutMix, lead to worse performance. Our experiments on ImageNet-1K dataset demonstrate that our specialized training strategy for compact models is applicable to various architectures, including GhostNetV2, MobileNetV2 and ShuffleNetV2. Specifically, equipped with our strategy, GhostNetV3 1.3$\\times$ achieves a top-1 accuracy of 79.1% with only 269M FLOPs and a latency of 14.46ms on mobile devices, surpassing its ordinarily trained counterpart by a large margin. Moreover, our observation can also be extended to object detection scenarios. PyTorch code and checkpoints can be found at https://github.com/huawei-noah/Efficient-AI-Backbones/tree/master/ghostnetv3_pytorch.","url_abs":"https://arxiv.org/abs/2404.11202v2","url_pdf":"https://arxiv.org/pdf/2404.11202v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ghostnetv3-exploring-the-training-strategies","repo_url":"https://github.com/james77777778/keras-image-models","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"cutmix","method_name":"CutMix"},{"method_slug":"depthwise-convolution","method_name":"Depthwise Convolution"},{"method_slug":"depthwise-separable-convolution","method_name":"Depthwise Separable Convolution"},{"method_slug":"inverted-residual-block","method_name":"Inverted Residual Block"},{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"},{"method_slug":"mixup","method_name":"Mixup"},{"method_slug":"pointwise-convolution","method_name":"Pointwise Convolution"},{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"GhostNetV3 1.6x","rank_in_archive_order":702,"of":1060,"metrics":{"Top 1 Accuracy":"80.4%","Top 5 Accuracy":"95.2"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"GhostNetV3 1.3x","rank_in_archive_order":775,"of":1060,"metrics":{"Top 1 Accuracy":"79.1%","Top 5 Accuracy":"94.5"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"GhostNetV3 1.0x","rank_in_archive_order":885,"of":1060,"metrics":{"Top 1 Accuracy":"77.1%","Top 5 Accuracy":"93.3"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"GhostNetV3 0.5x","rank_in_archive_order":1027,"of":1060,"metrics":{"Top 1 Accuracy":"69.4%","Top 5 Accuracy":"88.5"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2404.11202","atlas_url":"https://app.syntology.ai/?focus=2404.11202","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.11202"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/james77777778/keras-image-models","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":3},"by_repo_kind":{"listed":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1a05bf0ad9b109e7","entry":"apply_convmixer_block","repo":"james77777778/keras-image-models","repo_kind":"listed","path":"kimm/_src/models/convmixer.py","file_url":"https://github.com/james77777778/keras-image-models/blob/HEAD/kimm/_src/models/convmixer.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1a05bf0ad9b109e7"}},{"code_sha256_prefix":"ae798d1cc7be856b","entry":"apply_dense_transition_block","repo":"james77777778/keras-image-models","repo_kind":"listed","path":"kimm/_src/models/densenet.py","file_url":"https://github.com/james77777778/keras-image-models/blob/HEAD/kimm/_src/models/densenet.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ae798d1cc7be856b"}},{"code_sha256_prefix":"dbaa4c3c870c24a3","entry":"apply_inception_depthwise_conv2d","repo":"james77777778/keras-image-models","repo_kind":"listed","path":"kimm/_src/models/inception_next.py","file_url":"https://github.com/james77777778/keras-image-models/blob/HEAD/kimm/_src/models/inception_next.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"dbaa4c3c870c24a3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}