{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2508-20660","title":"CodecBench: A Comprehensive Benchmark for Acoustic and Semantic Evaluation","arxiv_id":"2508.20660","date":"2025-08-28","proceeding":null,"authors":["Ruifan Deng","Yitian Gong","Qinghui Gao","Luozhijie Jin","Qinyuan Cheng","Zhaoye Fei","Shimin Li","Xipeng Qiu"],"abstract":"With the rise of multimodal large language models (LLMs), audio codec plays an increasingly vital role in encoding audio into discrete tokens, enabling integration of audio into text-based LLMs. Current audio codec captures two types of information: acoustic and semantic. As audio codec is applied to diverse scenarios in speech language model , it needs to model increasingly complex information and adapt to varied contexts, such as scenarios with multiple speakers, background noise, or richer paralinguistic information. However, existing codec's own evaluation has been limited by simplistic metrics and scenarios, and existing benchmarks for audio codec are not designed for complex application scenarios, which limits the assessment performance on complex datasets for acoustic and semantic capabilities. We introduce CodecBench, a comprehensive evaluation dataset to assess audio codec performance from both acoustic and semantic perspectives across four data domains. Through this benchmark, we aim to identify current limitations, highlight future research directions, and foster advances in the development of audio codec. The codes are available at https://github.com/RayYuki/CodecBench.","url_abs":"https://arxiv.org/abs/2508.20660","url_pdf":"https://arxiv.org/pdf/2508.20660","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2508.20660","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2508.20660"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/RayYuki/CodecBench","reach":{"status":"ok"}}],"summary":{"unverified":6},"by_repo_kind":{"found_in_text":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"a6f9a3cde1bd5d05","entry":"compute_sisdr","repo":"RayYuki/CodecBench","repo_kind":"found_in_text","path":"reconstruct_evaluation/sdr_sisdr.py","file_url":"https://github.com/RayYuki/CodecBench/blob/HEAD/reconstruct_evaluation/sdr_sisdr.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a6f9a3cde1bd5d05"}},{"code_sha256_prefix":"4d537ee823124ddc","entry":"compute_stft_padding","repo":"RayYuki/CodecBench","repo_kind":"found_in_text","path":"reconstruct_evaluation/mel_loss.py","file_url":"https://github.com/RayYuki/CodecBench/blob/HEAD/reconstruct_evaluation/mel_loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4d537ee823124ddc"}},{"code_sha256_prefix":"13201779559c6a77","entry":"low_cut_filter","repo":"RayYuki/CodecBench","repo_kind":"found_in_text","path":"reconstruct_evaluation/mcd.py","file_url":"https://github.com/RayYuki/CodecBench/blob/HEAD/reconstruct_evaluation/mcd.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"13201779559c6a77"}},{"code_sha256_prefix":"bf52457fe3815f85","entry":"removeSilentFrames","repo":"RayYuki/CodecBench","repo_kind":"found_in_text","path":"reconstruct_evaluation/stoi.py","file_url":"https://github.com/RayYuki/CodecBench/blob/HEAD/reconstruct_evaluation/stoi.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bf52457fe3815f85"}},{"code_sha256_prefix":"35f1d84f20aa052f","entry":"sqrt_hann_window","repo":"RayYuki/CodecBench","repo_kind":"found_in_text","path":"reconstruct_evaluation/mel_loss.py","file_url":"https://github.com/RayYuki/CodecBench/blob/HEAD/reconstruct_evaluation/mel_loss.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"35f1d84f20aa052f"}},{"code_sha256_prefix":"62c154dc717f72eb","entry":"stoi","repo":"RayYuki/CodecBench","repo_kind":"found_in_text","path":"reconstruct_evaluation/stoi.py","file_url":"https://github.com/RayYuki/CodecBench/blob/HEAD/reconstruct_evaluation/stoi.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"62c154dc717f72eb"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"eess.AS","source":"arxiv_api"},"syntology_extracted_results":null}