{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/efficient-autoregressive-audio-modeling-via","title":"Efficient Autoregressive Audio Modeling via Next-Scale Prediction","arxiv_id":"2408.09027","date":"2024-08-16","proceeding":null,"authors":["Kai Qiu","Xiang Li","Hao Chen","Jie Sun","Jinglu Wang","Zhe Lin","Marios Savvides","Bhiksha Raj"],"abstract":"Audio generation has achieved remarkable progress with the advance of sophisticated generative models, such as diffusion models (DMs) and autoregressive (AR) models. However, due to the naturally significant sequence length of audio, the efficiency of audio generation remains an essential issue to be addressed, especially for AR models that are incorporated in large language models (LLMs). In this paper, we analyze the token length of audio tokenization and propose a novel \\textbf{S}cale-level \\textbf{A}udio \\textbf{T}okenizer (SAT), with improved residual quantization. Based on SAT, a scale-level \\textbf{A}coustic \\textbf{A}uto\\textbf{R}egressive (AAR) modeling framework is further proposed, which shifts the next-token AR prediction to next-scale AR prediction, significantly reducing the training cost and inference time. To validate the effectiveness of the proposed approach, we comprehensively analyze design choices and demonstrate the proposed AAR framework achieves a remarkable \\textbf{35}$\\times$ faster inference speed and +\\textbf{1.33} Fr\\'echet Audio Distance (FAD) against baselines on the AudioSet benchmark. Code: \\url{https://github.com/qiuk2/AAR}.","url_abs":"https://arxiv.org/abs/2408.09027v2","url_pdf":"https://arxiv.org/pdf/2408.09027v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"efficient-autoregressive-audio-modeling-via","repo_url":"https://github.com/qiuk2/aar","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"audio-generation","task_name":"Audio Generation"},{"task_slug":"fad","task_name":"FAD"},{"task_slug":"prediction","task_name":"Prediction"},{"task_slug":"quantization","task_name":"Quantization"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2408.09027","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2408.09027"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/qiuk2/aar","reach":null}],"summary":{"ran_honours":1,"ran_fixture":1},"by_repo_kind":{"official":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":2,"samples":[{"code_sha256_prefix":"d09dbeb0d659afef","entry":"extract_step","repo":"qiuk2/aar","repo_kind":"official","path":"train_AAR_mpi.py","file_url":"https://github.com/qiuk2/aar/blob/HEAD/train_AAR_mpi.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d09dbeb0d659afef"}},{"code_sha256_prefix":"5950cd82bc8a8836","entry":"inference","repo":"qiuk2/aar","repo_kind":"official","path":"inference_AAR.py","file_url":"https://github.com/qiuk2/aar/blob/HEAD/inference_AAR.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5950cd82bc8a8836"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}