{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/automatic-joint-structured-pruning-and","title":"Automatic Joint Structured Pruning and Quantization for Efficient Neural Network Training and Compression","arxiv_id":"2502.16638","date":"2025-02-23","proceeding":"CVPR 2025 1","authors":["Xiaoyi Qu","David Aponte","Colby Banbury","Daniel P. Robinson","Tianyu Ding","Kazuhito Koishida","Ilya Zharkov","Tianyi Chen"],"abstract":"Structured pruning and quantization are fundamental techniques used to reduce the size of deep neural networks (DNNs) and typically are applied independently. Applying these techniques jointly via co-optimization has the potential to produce smaller, high-quality models. However, existing joint schemes are not widely used because of (1) engineering difficulties (complicated multi-stage processes), (2) black-box optimization (extensive hyperparameter tuning to control the overall compression), and (3) insufficient architecture generalization. To address these limitations, we present the framework GETA, which automatically and efficiently performs joint structured pruning and quantization-aware training on any DNNs. GETA introduces three key innovations: (i) a quantization-aware dependency graph (QADG) that constructs a pruning search space for generic quantization-aware DNN, (ii) a partially projected stochastic gradient method that guarantees layerwise bit constraints are satisfied, and (iii) a new joint learning strategy that incorporates interpretable relationships between pruning and quantization. We present numerical experiments on both convolutional neural networks and transformer architectures that show that our approach achieves competitive (often superior) performance compared to existing joint pruning and quantization methods.","url_abs":"https://arxiv.org/abs/2502.16638v1","url_pdf":"https://arxiv.org/pdf/2502.16638v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"automatic-joint-structured-pruning-and","repo_url":"https://github.com/microsoft/geta","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"jax","reach":null}],"tasks":[{"task_slug":"efficient-neural-network","task_name":"Efficient Neural Network"},{"task_slug":"quantization","task_name":"Quantization"}],"methods":[{"method_slug":"pruning","method_name":"Pruning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2502.16638","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2502.16638"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/microsoft/GETA","reach":null}],"summary":{"ran":1,"unverified":4},"by_repo_kind":{"official":{"samples":5,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"fe60ffa8a38b735c","entry":"BaseOptimizer","repo":"microsoft/GETA","repo_kind":"official","path":"only_train_once/optimizer/geta.py","file_url":"https://github.com/microsoft/GETA/blob/HEAD/only_train_once/optimizer/geta.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fe60ffa8a38b735c"}},{"code_sha256_prefix":"6a32f11a39342a9c","entry":"BaseHybridSparseOptimizer","repo":"microsoft/GETA","repo_kind":"official","path":"only_train_once/optimizer/geta.py","file_url":"https://github.com/microsoft/GETA/blob/HEAD/only_train_once/optimizer/geta.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6a32f11a39342a9c"}},{"code_sha256_prefix":"6c2adeebfa94e68e","entry":"GETA","repo":"microsoft/GETA","repo_kind":"official","path":"only_train_once/optimizer/geta.py","file_url":"https://github.com/microsoft/GETA/blob/HEAD/only_train_once/optimizer/geta.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6c2adeebfa94e68e"}},{"code_sha256_prefix":"d00b5cb6364f2e2d","entry":"SparseOptimizerMetrics","repo":"microsoft/GETA","repo_kind":"official","path":"only_train_once/optimizer/geta.py","file_url":"https://github.com/microsoft/GETA/blob/HEAD/only_train_once/optimizer/geta.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d00b5cb6364f2e2d"}},{"code_sha256_prefix":"b3111868968fda3e","entry":"calculate_importance_score","repo":"microsoft/GETA","repo_kind":"official","path":"only_train_once/optimizer/geta.py","file_url":"https://github.com/microsoft/GETA/blob/HEAD/only_train_once/optimizer/geta.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b3111868968fda3e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}