{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2605-04084","title":"FASQ: Flexible Accelerated Subspace Quantization for Calibration-Free LLM Compression","arxiv_id":"2605.04084","date":"2026-04-22","proceeding":null,"authors":["Ye Qiao","Yian Wang","Zhiheng Chen","Hyoukjun Kwon","Sitao Huang"],"abstract":"Compressing large language models (LLMs) for deployment on commodity GPUs remains challenging: conventional scalar quantization is limited to fixed bit-widths (e.g., 8/4/3-bit), offers only a few discrete compression points, and typically requires calibration data. We present FASQ (Flexible Accelerated Subspace Quantization), a calibration-free framework that applies product quantization to LLM weight matrices. By tuning two parameters, sub-vector size and codebook cardinality, FASQ exposes a continuous design space spanning 27-49% of the original FP16 model size, filling compression gaps that fixed-bit schemes cannot reach. On Meta-Llama-3-8B, FASQ surpasses 4-bit GPTQ and AWQ in accuracy (67.1-67.7 avg.) at 37-42% model size, with consistent results on Qwen3-8B and Qwen3.5-9B-Base. To make product quantization practical at inference time, we design custom CUDA kernels: a LUT-free direct-compute GEMV for decode and an output-stationary double-buffered LUT GEMM for prefill, both with split-K parallelism. On an RTX~3090, FASQ achieves 45.2 tok/s decode at effective 4-bit (2.56x memory reduction) and 51.8 tok/s at effective 3-bit (2.80x), both surpassing FP16 tensor-core performance (43.9 tok/s) and delivering 1.6 to 1.8x the throughput of AWQ, 2.5 to 2.5x of GPTQ, and 4.3 to 5x of RTN. FASQ is the only compressed method that accelerates decode beyond FP16, offering calibration-free compression, continuous size-quality trade-offs, and real-time inference on a single consumer GPU.","url_abs":"https://arxiv.org/abs/2605.04084","url_pdf":"https://arxiv.org/pdf/2605.04084","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2605.04084","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2605.04084"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/mit-han-lab/llm-awq","reach":null},{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/mit-han-lab/smoothquant","reach":null},{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/ist-daslab/gptq","reach":null}],"summary":{"ran_fixture":1,"ran_draft_wrong":4,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":7,"ran":5,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"41c57e55d21c4d96","entry":"quantize","repo":"ist-daslab/gptq","repo_kind":"found_in_text","path":"quant.py","file_url":"https://github.com/ist-daslab/gptq/blob/HEAD/quant.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"41c57e55d21c4d96"}},{"code_sha256_prefix":"aa2bd5edebc9b6ff","entry":"quantize_activation_per_tensor_absmax","repo":"mit-han-lab/smoothquant","repo_kind":"found_in_text","path":"smoothquant/fake_quant.py","file_url":"https://github.com/mit-han-lab/smoothquant/blob/HEAD/smoothquant/fake_quant.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa2bd5edebc9b6ff"}},{"code_sha256_prefix":"c1cb961b4f5cb75c","entry":"quantize_activation_per_token_absmax","repo":"mit-han-lab/smoothquant","repo_kind":"found_in_text","path":"smoothquant/fake_quant.py","file_url":"https://github.com/mit-han-lab/smoothquant/blob/HEAD/smoothquant/fake_quant.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c1cb961b4f5cb75c"}},{"code_sha256_prefix":"770da8497b2ff4ba","entry":"quantize_weight_per_channel_absmax","repo":"mit-han-lab/smoothquant","repo_kind":"found_in_text","path":"smoothquant/fake_quant.py","file_url":"https://github.com/mit-han-lab/smoothquant/blob/HEAD/smoothquant/fake_quant.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"770da8497b2ff4ba"}},{"code_sha256_prefix":"dd3493503cb95888","entry":"quantize_weight_per_tensor_absmax","repo":"mit-han-lab/smoothquant","repo_kind":"found_in_text","path":"smoothquant/fake_quant.py","file_url":"https://github.com/mit-han-lab/smoothquant/blob/HEAD/smoothquant/fake_quant.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dd3493503cb95888"}},{"code_sha256_prefix":"fc31370a0c6e1aaf","entry":"Quantizer","repo":"ist-daslab/gptq","repo_kind":"found_in_text","path":"quant.py","file_url":"https://github.com/ist-daslab/gptq/blob/HEAD/quant.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fc31370a0c6e1aaf"}},{"code_sha256_prefix":"fcb15f76a1133e67","entry":"W8A8Linear","repo":"mit-han-lab/smoothquant","repo_kind":"found_in_text","path":"smoothquant/fake_quant.py","file_url":"https://github.com/mit-han-lab/smoothquant/blob/HEAD/smoothquant/fake_quant.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fcb15f76a1133e67"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}