{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-clustering-discriminative-embeddings-for","title":"Deep clustering: Discriminative embeddings for segmentation and separation","arxiv_id":"1508.04306","date":"2015-08-18","proceeding":null,"authors":["John R. Hershey","Zhuo Chen","Jonathan Le Roux","Shinji Watanabe"],"abstract":"We address the problem of acoustic source separation in a deep learning\nframework we call \"deep clustering.\" Rather than directly estimating signals or\nmasking functions, we train a deep network to produce spectrogram embeddings\nthat are discriminative for partition labels given in training data. Previous\ndeep network approaches provide great advantages in terms of learning power and\nspeed, but previously it has been unclear how to use them to separate signals\nin a class-independent way. In contrast, spectral clustering approaches are\nflexible with respect to the classes and number of items to be segmented, but\nit has been unclear how to leverage the learning power and speed of deep\nnetworks. To obtain the best of both worlds, we use an objective function that\nto train embeddings that yield a low-rank approximation to an ideal pairwise\naffinity matrix, in a class-independent way. This avoids the high cost of\nspectral factorization and instead produces compact clusters that are amenable\nto simple clustering methods. The segmentations are therefore implicitly\nencoded in the embeddings, and can be \"decoded\" by clustering. Preliminary\nexperiments show that the proposed method can separate speech: when trained on\nspectrogram features containing mixtures of two speakers, and tested on\nmixtures of a held-out set of speakers, it can infer masking functions that\nimprove signal quality by around 6dB. We show that the model can generalize to\nthree-speaker mixtures despite training only on two-speaker mixtures. The\nframework can be used without class labels, and therefore has the potential to\nbe trained on a diverse set of sound types, and to generalize to novel sources.\nWe hope that future work will lead to segmentation of arbitrary sounds, with\nextensions to microphone array methods as well as image segmentation and other\ndomains.","url_abs":"http://arxiv.org/abs/1508.04306v1","url_pdf":"http://arxiv.org/pdf/1508.04306v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-clustering-discriminative-embeddings-for","repo_url":"https://github.com/abugler/SMLFinalProject","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"deep-clustering-discriminative-embeddings-for","repo_url":"https://github.com/fgnt/mms_msg","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"deep-clustering-discriminative-embeddings-for","repo_url":"https://github.com/ishandutta2007/Speech-Denoising-Landscape","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"deep-clustering-discriminative-embeddings-for","repo_url":"https://github.com/jack20951948/Deep-Clustering","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"deep-clustering-discriminative-embeddings-for","repo_url":"https://github.com/lordet01/deep_clustering","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"deep-clustering-discriminative-embeddings-for","repo_url":"https://github.com/mpariente/asteroid","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"deep-clustering-discriminative-embeddings-for","repo_url":"https://github.com/tsian077/Deep_Clustering_EfficientNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"deep-clustering-discriminative-embeddings-for","repo_url":"https://github.com/JusperLee/Deep-Clustering-for-Speech-Separation","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"clustering","task_name":"Clustering"},{"task_slug":"deep-clustering","task_name":"Deep Clustering"},{"task_slug":"image-segmentation","task_name":"Image Segmentation"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"},{"task_slug":"speech-separation","task_name":"Speech Separation"}],"methods":[{"method_slug":"speed","method_name":"SPEED"},{"method_slug":"spectral-clustering","method_name":"Spectral Clustering"}],"datasets_introduced":[{"slug":"wsj0-2mix-1","name":"WSJ0-2mix","full_name":""}],"methods_introduced":[],"results":[{"leaderboard":"/sota/speech-separation-on-wsj0-2mix","task":"Speech Separation","dataset":"WSJ0-2mix","model":"Deep Clustering ++","rank_in_archive_order":40,"of":40,"metrics":{"SI-SDRi":"10.8"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1508.04306","atlas_url":"https://app.syntology.ai/?focus=1508.04306","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1508.04306"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mpariente/asteroid","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ishandutta2007/Speech-Denoising-Landscape","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/abugler/SMLFinalProject","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tsian077/Deep_Clustering_EfficientNet","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/JusperLee/Deep-Clustering-for-Speech-Separation","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lordet01/deep_clustering","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jack20951948/Deep-Clustering","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/fgnt/mms_msg","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":2,"unverified":1},"by_repo_kind":{"listed":{"samples":3,"ran":2,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"38882f5702be3e74","entry":"round_filters","repo":"tsian077/Deep_Clustering_EfficientNet","repo_kind":"listed","path":"DeWave/model.py","file_url":"https://github.com/tsian077/Deep_Clustering_EfficientNet/blob/HEAD/DeWave/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"38882f5702be3e74"}},{"code_sha256_prefix":"30fc65e29d79bc63","entry":"round_repeats","repo":"tsian077/Deep_Clustering_EfficientNet","repo_kind":"listed","path":"DeWave/model.py","file_url":"https://github.com/tsian077/Deep_Clustering_EfficientNet/blob/HEAD/DeWave/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"30fc65e29d79bc63"}},{"code_sha256_prefix":"a6f6fda8ee5763b1","entry":"stft","repo":"jack20951948/Deep-Clustering","repo_kind":"listed","path":"audio_test.py","file_url":"https://github.com/jack20951948/Deep-Clustering/blob/HEAD/audio_test.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a6f6fda8ee5763b1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}