{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2605-06407","title":"WavCube: Unifying Speech Representation for Understanding and Generation via Semantic-Acoustic Joint Modeling","arxiv_id":"2605.06407","date":"2026-05-07","proceeding":null,"authors":["Guanrou Yang","Tian Tan","Qian Chen","Zhikang Niu","Yakun Song","Ziyang Ma","Yushen Chen","Zeyu Xie","Tianrui Wang","Yifan Yang","Wenxi Chen","Qi Chen","Wenrui Liu","Shan Yang","Xie Chen"],"abstract":"Integrating speech understanding and generation is a pivotal step toward building unified speech models. However, the different representations required for these two tasks currently pose significant compatibility challenges. Typically, semantics-oriented features are learned from self-supervised learning (SSL), and acoustic-oriented features from reconstruction. Such fragmented representations hinder the realization of truly unified speech systems. We present WavCube, a compact continuous latent derived from an SSL speech encoder that simultaneously supports speech understanding, reconstruction, and generation. WavCube employs a two-stage training scheme. Stage 1 trains a semantic bottleneck to filter off-manifold redundancy that makes raw SSL features intractable for diffusion. Stage 2 injects fine-grained acoustic details via end-to-end reconstruction, while a semantic anchoring loss ensures the representation remains grounded within its original semantic manifold. Comprehensive experiments show that WavCube closely approaches WavLM performance on SUPERB despite an 8x dimensional compression, attains reconstruction quality on par with existing acoustic representations, delivers state-of-the-art zero-shot TTS performance with markedly faster training convergence, and excels in speech enhancement, separation, and voice conversion tasks on the SUPERB-SG benchmark. Systematic ablations reveal that WavCube's two-stage recipe resolves two intrinsic flaws of SSL features for generative modeling, paving the way for future unified speech systems. Codes and checkpoints are available at https://github.com/yanghaha0908/WavCube.","url_abs":"https://arxiv.org/abs/2605.06407","url_pdf":"https://arxiv.org/pdf/2605.06407","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2605.06407","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2605.06407"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/yanghaha0908/WavCube","reach":null}],"summary":{"ran":7,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":8,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ad9a15f964a5b45f","entry":"characterize","repo":"yanghaha0908/WavCube","repo_kind":"found_in_text","path":"evaluation/compute_wer.py","file_url":"https://github.com/yanghaha0908/WavCube/blob/HEAD/evaluation/compute_wer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ad9a15f964a5b45f"}},{"code_sha256_prefix":"10ae85266c7ccbf9","entry":"load_feature","repo":"yanghaha0908/WavCube","repo_kind":"found_in_text","path":"feature_to_wav.py","file_url":"https://github.com/yanghaha0908/WavCube/blob/HEAD/feature_to_wav.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"10ae85266c7ccbf9"}},{"code_sha256_prefix":"e14967660d98560c","entry":"normalize","repo":"yanghaha0908/WavCube","repo_kind":"found_in_text","path":"evaluation/compute_wer.py","file_url":"https://github.com/yanghaha0908/WavCube/blob/HEAD/evaluation/compute_wer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e14967660d98560c"}},{"code_sha256_prefix":"5923f41e711c0b69","entry":"safe_log","repo":"yanghaha0908/WavCube","repo_kind":"found_in_text","path":"vocos/modules.py","file_url":"https://github.com/yanghaha0908/WavCube/blob/HEAD/vocos/modules.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5923f41e711c0b69"}},{"code_sha256_prefix":"32ade8e8e8927d3e","entry":"stripoff_tags","repo":"yanghaha0908/WavCube","repo_kind":"found_in_text","path":"evaluation/compute_wer.py","file_url":"https://github.com/yanghaha0908/WavCube/blob/HEAD/evaluation/compute_wer.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"32ade8e8e8927d3e"}},{"code_sha256_prefix":"f1abe87534e9b4b5","entry":"symexp","repo":"yanghaha0908/WavCube","repo_kind":"found_in_text","path":"vocos/modules.py","file_url":"https://github.com/yanghaha0908/WavCube/blob/HEAD/vocos/modules.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f1abe87534e9b4b5"}},{"code_sha256_prefix":"3acb919a94ac1a19","entry":"symlog","repo":"yanghaha0908/WavCube","repo_kind":"found_in_text","path":"vocos/modules.py","file_url":"https://github.com/yanghaha0908/WavCube/blob/HEAD/vocos/modules.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3acb919a94ac1a19"}},{"code_sha256_prefix":"bea543d0e76dfc84","entry":"ECAPA_TDNN_SMALL","repo":"yanghaha0908/WavCube","repo_kind":"found_in_text","path":"evaluation/ecapa_tdnn.py","file_url":"https://github.com/yanghaha0908/WavCube/blob/HEAD/evaluation/ecapa_tdnn.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bea543d0e76dfc84"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}