{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/self-supervised-speech-quality-estimation-and","title":"Self-Supervised Speech Quality Estimation and Enhancement Using Only Clean Speech","arxiv_id":"2402.16321","date":"2024-02-26","proceeding":null,"authors":["Szu-Wei Fu","Kuo-Hsuan Hung","Yu Tsao","Yu-Chiang Frank Wang"],"abstract":"Speech quality estimation has recently undergone a paradigm shift from human-hearing expert designs to machine-learning models. However, current models rely mainly on supervised learning, which is time-consuming and expensive for label collection. To solve this problem, we propose VQScore, a self-supervised metric for evaluating speech based on the quantization error of a vector-quantized-variational autoencoder (VQ-VAE). The training of VQ-VAE relies on clean speech; hence, large quantization errors can be expected when the speech is distorted. To further improve correlation with real quality scores, domain knowledge of speech processing is incorporated into the model design. We found that the vector quantization mechanism could also be used for self-supervised speech enhancement (SE) model training. To improve the robustness of the encoder for SE, a novel self-distillation mechanism combined with adversarial training is introduced. In summary, the proposed speech quality estimation method and enhancement models require only clean speech for training without any label requirements. Experimental results show that the proposed VQScore and enhancement model are competitive with supervised baselines. The code will be released after publication.","url_abs":"https://arxiv.org/abs/2402.16321v1","url_pdf":"https://arxiv.org/pdf/2402.16321v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"self-supervised-speech-quality-estimation-and","repo_url":"https://github.com/JasonSWFu/VQscore","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"quantization","task_name":"Quantization"},{"task_slug":"speech-enhancement","task_name":"Speech Enhancement"}],"methods":[{"method_slug":"vq-vae","method_name":"VQ-VAE"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.16321","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.16321"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/JasonSWFu/VQscore","reach":null}],"summary":{"ran":3,"ran_draft_wrong":2,"unverified":6},"by_repo_kind":{"official":{"samples":11,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":11,"samples":[{"code_sha256_prefix":"b8a0165c23a73cb3","entry":"CNN_1D_decoder_QE","repo":"JasonSWFu/VQscore","repo_kind":"official","path":"models/VQVAE_models.py","file_url":"https://github.com/JasonSWFu/VQscore/blob/HEAD/models/VQVAE_models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b8a0165c23a73cb3"}},{"code_sha256_prefix":"9e5d56abc2a08a53","entry":"CNN_1D_encoder_QE","repo":"JasonSWFu/VQscore","repo_kind":"official","path":"models/VQVAE_models.py","file_url":"https://github.com/JasonSWFu/VQscore/blob/HEAD/models/VQVAE_models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9e5d56abc2a08a53"}},{"code_sha256_prefix":"56de883df56b1fbb","entry":"cos_loss","repo":"JasonSWFu/VQscore","repo_kind":"official","path":"inference.py","file_url":"https://github.com/JasonSWFu/VQscore/blob/HEAD/inference.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"56de883df56b1fbb"}},{"code_sha256_prefix":"fb5ef568d9dbf7b5","entry":"gumbel_sample","repo":"JasonSWFu/VQscore","repo_kind":"official","path":"models/VQVAE_models.py","file_url":"https://github.com/JasonSWFu/VQscore/blob/HEAD/models/VQVAE_models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fb5ef568d9dbf7b5"}},{"code_sha256_prefix":"c05fc59409ba925c","entry":"stft_magnitude","repo":"JasonSWFu/VQscore","repo_kind":"official","path":"inference.py","file_url":"https://github.com/JasonSWFu/VQscore/blob/HEAD/inference.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c05fc59409ba925c"}},{"code_sha256_prefix":"1ccf86e8795fed3b","entry":"CNN_1D_quantizer_QE","repo":"JasonSWFu/VQscore","repo_kind":"official","path":"models/VQVAE_models.py","file_url":"https://github.com/JasonSWFu/VQscore/blob/HEAD/models/VQVAE_models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1ccf86e8795fed3b"}},{"code_sha256_prefix":"f8e325b49ff91a19","entry":"CosineSimCodebook","repo":"JasonSWFu/VQscore","repo_kind":"official","path":"models/VQVAE_models.py","file_url":"https://github.com/JasonSWFu/VQscore/blob/HEAD/models/VQVAE_models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f8e325b49ff91a19"}},{"code_sha256_prefix":"252f6a2a01339c05","entry":"EuclideanCodebook","repo":"JasonSWFu/VQscore","repo_kind":"official","path":"models/VQVAE_models.py","file_url":"https://github.com/JasonSWFu/VQscore/blob/HEAD/models/VQVAE_models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"252f6a2a01339c05"}},{"code_sha256_prefix":"e7029fcc1359a25e","entry":"VQVAE_QE","repo":"JasonSWFu/VQscore","repo_kind":"official","path":"models/VQVAE_models.py","file_url":"https://github.com/JasonSWFu/VQscore/blob/HEAD/models/VQVAE_models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e7029fcc1359a25e"}},{"code_sha256_prefix":"23568f6dd68bbe9b","entry":"VectorQuantize","repo":"JasonSWFu/VQscore","repo_kind":"official","path":"models/VQVAE_models.py","file_url":"https://github.com/JasonSWFu/VQscore/blob/HEAD/models/VQVAE_models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"23568f6dd68bbe9b"}},{"code_sha256_prefix":"fb486c5b29adc71b","entry":"resynthesize","repo":"JasonSWFu/VQscore","repo_kind":"official","path":"inference.py","file_url":"https://github.com/JasonSWFu/VQscore/blob/HEAD/inference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fb486c5b29adc71b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}