{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/outlier-suppression-accurate-quantization-of","title":"Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling","arxiv_id":"2304.09145","date":"2023-04-18","proceeding":null,"authors":["Xiuying Wei","Yunchen Zhang","Yuhang Li","Xiangguo Zhang","Ruihao Gong","Jinyang Guo","Xianglong Liu"],"abstract":"Post-training quantization~(PTQ) of transformer language models faces significant challenges due to the existence of detrimental outliers in activations. We observe that these outliers are concentrated in specific channels and are asymmetric across channels. To address this issue, we propose the Outlier Suppression+~(OS+) framework, which contains the channel-wise shifting for asymmetry and channel-wise scaling for concentration. We show that these operations can be seamlessly migrated into subsequent modules while maintaining equivalence. Second, we propose a fast and stable scheme to calculate effective shifting and scaling values. The channel-wise shifting aligns the center of each channel for removal of outlier asymmetry. The channel-wise scaling quantitatively evaluates changes brought by migration and quantization for better quantization burden balance. We validate our OS+ under both standard and fine-grained quantization settings with models including BERT, OPT, BLOOM, BLOOMZ, and LLaMA. Comprehensive results across various tasks demonstrate the superiority of our approach. Especially, with standard quantization, OS+ can achieve near-floating-point performance on both small models and large language models on 8-bit and 6-bit. Besides, we establish a new state-of-the-art for 4-bit BERT with 15.5\\% improvement. Our code is available at \\url{https://github.com/ModelTC/Outlier_Suppression_Plus}.","url_abs":"https://arxiv.org/abs/2304.09145v3","url_pdf":"https://arxiv.org/pdf/2304.09145v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"outlier-suppression-accurate-quantization-of","repo_url":"https://github.com/modeltc/outlier_suppression_plus","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"quantization","task_name":"Quantization"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"attention-dropout","method_name":"Attention Dropout"},{"method_slug":"bert","method_name":"BERT"},{"method_slug":"bloom","method_name":"BLOOM"},{"method_slug":"bloomz","method_name":"BLOOMZ"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-linear-decay","method_name":"Linear Warmup With Linear Decay"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"opt","method_name":"OPT"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"weight-decay","method_name":"Weight Decay"},{"method_slug":"wordpiece","method_name":"WordPiece"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2304.09145","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2304.09145"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/EleutherAI/lm-evaluation-harness","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/modeltc/outlier_suppression_plus","reach":null}],"summary":{"ran":2,"ran_fixture":1,"ran_draft_wrong":2,"unverified":2},"by_repo_kind":{"official":{"samples":7,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"618d2924baa4dc1d","entry":"MinMaxObserver","repo":"modeltc/outlier_suppression_plus","repo_kind":"official","path":"quant_transformer/quantization/migration.py","file_url":"https://github.com/modeltc/outlier_suppression_plus/blob/HEAD/quant_transformer/quantization/migration.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"618d2924baa4dc1d"}},{"code_sha256_prefix":"2f6f74a2cba77dd3","entry":"ObserverBase","repo":"modeltc/outlier_suppression_plus","repo_kind":"official","path":"quant_transformer/quantization/migration.py","file_url":"https://github.com/modeltc/outlier_suppression_plus/blob/HEAD/quant_transformer/quantization/migration.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2f6f74a2cba77dd3"}},{"code_sha256_prefix":"90fb18c1f9adc9e2","entry":"_transform_to_ch_axis","repo":"modeltc/outlier_suppression_plus","repo_kind":"official","path":"quant_transformer/quantization/migration.py","file_url":"https://github.com/modeltc/outlier_suppression_plus/blob/HEAD/quant_transformer/quantization/migration.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"90fb18c1f9adc9e2"}},{"code_sha256_prefix":"c0dfc9372ddd11ec","entry":"fake_quantize_per_channel_affine","repo":"modeltc/outlier_suppression_plus","repo_kind":"official","path":"quant_transformer/quantization/migration.py","file_url":"https://github.com/modeltc/outlier_suppression_plus/blob/HEAD/quant_transformer/quantization/migration.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c0dfc9372ddd11ec"}},{"code_sha256_prefix":"f5f3045a5c318d7c","entry":"fake_quantize_per_tensor_affine","repo":"modeltc/outlier_suppression_plus","repo_kind":"official","path":"quant_transformer/quantization/migration.py","file_url":"https://github.com/modeltc/outlier_suppression_plus/blob/HEAD/quant_transformer/quantization/migration.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f5f3045a5c318d7c"}},{"code_sha256_prefix":"8b470374c5bcffcf","entry":"Migrator1DRangeSearch","repo":"modeltc/outlier_suppression_plus","repo_kind":"official","path":"quant_transformer/quantization/migration.py","file_url":"https://github.com/modeltc/outlier_suppression_plus/blob/HEAD/quant_transformer/quantization/migration.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8b470374c5bcffcf"}},{"code_sha256_prefix":"0b04ff29f1122a99","entry":"MigratorBase","repo":"modeltc/outlier_suppression_plus","repo_kind":"official","path":"quant_transformer/quantization/migration.py","file_url":"https://github.com/modeltc/outlier_suppression_plus/blob/HEAD/quant_transformer/quantization/migration.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0b04ff29f1122a99"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}