{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/magicodec-simple-masked-gaussian-injected","title":"MagiCodec: Simple Masked Gaussian-Injected Codec for High-Fidelity Reconstruction and Generation","arxiv_id":"2506.00385","date":"2025-05-31","proceeding":null,"authors":["Yakun Song","Jiawei Chen","Xiaobin Zhuang","Chenpeng Du","Ziyang Ma","Jian Wu","Jian Cong","Dongya Jia","Zhuo Chen","Yuping Wang","Yuxuan Wang","Xie Chen"],"abstract":"Neural audio codecs have made significant strides in efficiently mapping raw audio waveforms into discrete token representations, which are foundational for contemporary audio generative models. However, most existing codecs are optimized primarily for reconstruction quality, often at the expense of the downstream modelability of the encoded tokens. Motivated by the need to overcome this bottleneck, we introduce $\\textbf{MagiCodec}$, a novel single-layer, streaming Transformer-based audio codec. MagiCodec is designed with a multistage training pipeline that incorporates Gaussian noise injection and latent regularization, explicitly targeting the enhancement of semantic expressiveness in the generated codes while preserving high reconstruction fidelity. We analytically derive the effect of noise injection in the frequency domain, demonstrating its efficacy in attenuating high-frequency components and fostering robust tokenization. Extensive experimental evaluations show that MagiCodec surpasses state-of-the-art codecs in both reconstruction quality and downstream tasks. Notably, the tokens produced by MagiCodec exhibit Zipf-like distributions, as observed in natural languages, thereby improving compatibility with language-model-based generative architectures. The code and pre-trained models are available at https://github.com/Ereboas/MagiCodec.","url_abs":"https://arxiv.org/abs/2506.00385v1","url_pdf":"https://arxiv.org/pdf/2506.00385v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"magicodec-simple-masked-gaussian-injected","repo_url":"https://github.com/ereboas/magicodec","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2506.00385","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2506.00385"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/Ereboas/MagiCodec","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"ran_violates":1,"unverified":6},"by_repo_kind":{"official":{"samples":8,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"0bda6e53e10d00b2","entry":"bias_gelu","repo":"Ereboas/MagiCodec","repo_kind":"official","path":"components/ops/activations.py","file_url":"https://github.com/Ereboas/MagiCodec/blob/HEAD/components/ops/activations.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0bda6e53e10d00b2"}},{"code_sha256_prefix":"14a84c2cbfebc413","entry":"precompute_freqs_cis","repo":"Ereboas/MagiCodec","repo_kind":"official","path":"roformer.py","file_url":"https://github.com/Ereboas/MagiCodec/blob/HEAD/roformer.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"14a84c2cbfebc413"}},{"code_sha256_prefix":"06a68b038a2bc78d","entry":"bias_gelu_back","repo":"Ereboas/MagiCodec","repo_kind":"official","path":"components/ops/activations.py","file_url":"https://github.com/Ereboas/MagiCodec/blob/HEAD/components/ops/activations.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"06a68b038a2bc78d"}},{"code_sha256_prefix":"58e6542897e7d424","entry":"maybe_contiguous","repo":"Ereboas/MagiCodec","repo_kind":"official","path":"components/ops/flash_attn_2_3_interface.py","file_url":"https://github.com/Ereboas/MagiCodec/blob/HEAD/components/ops/flash_attn_2_3_interface.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"58e6542897e7d424"}},{"code_sha256_prefix":"9da99d54baee61a5","entry":"modulate","repo":"Ereboas/MagiCodec","repo_kind":"official","path":"components/block.py","file_url":"https://github.com/Ereboas/MagiCodec/blob/HEAD/components/block.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9da99d54baee61a5"}},{"code_sha256_prefix":"9112633659fb264d","entry":"new_gelu","repo":"Ereboas/MagiCodec","repo_kind":"official","path":"components/ops/activations.py","file_url":"https://github.com/Ereboas/MagiCodec/blob/HEAD/components/ops/activations.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9112633659fb264d"}},{"code_sha256_prefix":"3c9fe8b8927b497e","entry":"pad_input","repo":"Ereboas/MagiCodec","repo_kind":"official","path":"components/ops/padding.py","file_url":"https://github.com/Ereboas/MagiCodec/blob/HEAD/components/ops/padding.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3c9fe8b8927b497e"}},{"code_sha256_prefix":"b4ca9ddf6b6d4cad","entry":"unpad_input","repo":"Ereboas/MagiCodec","repo_kind":"official","path":"components/ops/padding.py","file_url":"https://github.com/Ereboas/MagiCodec/blob/HEAD/components/ops/padding.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b4ca9ddf6b6d4cad"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}