{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/separate-anything-you-describe","title":"Separate Anything You Describe","arxiv_id":"2308.05037","date":"2023-08-09","proceeding":null,"authors":["Xubo Liu","Qiuqiang Kong","Yan Zhao","Haohe Liu","Yi Yuan","Yuzhuo Liu","Rui Xia","Yuxuan Wang","Mark D. Plumbley","Wenwu Wang"],"abstract":"Language-queried audio source separation (LASS) is a new paradigm for computational auditory scene analysis (CASA). LASS aims to separate a target sound from an audio mixture given a natural language query, which provides a natural and scalable interface for digital audio applications. Recent works on LASS, despite attaining promising separation performance on specific sources (e.g., musical instruments, limited classes of audio events), are unable to separate audio concepts in the open domain. In this work, we introduce AudioSep, a foundation model for open-domain audio source separation with natural language queries. We train AudioSep on large-scale multimodal datasets and extensively evaluate its capabilities on numerous tasks including audio event separation, musical instrument separation, and speech enhancement. AudioSep demonstrates strong separation performance and impressive zero-shot generalization ability using audio captions or text labels as queries, substantially outperforming previous audio-queried and language-queried sound separation models. For reproducibility of this work, we will release the source code, evaluation benchmark and pre-trained model at: https://github.com/Audio-AGI/AudioSep.","url_abs":"https://arxiv.org/abs/2308.05037v3","url_pdf":"https://arxiv.org/pdf/2308.05037v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"separate-anything-you-describe","repo_url":"https://github.com/audio-agi/audiosep","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"audio-source-separation","task_name":"Audio Source Separation"},{"task_slug":"natural-language-queries","task_name":"Natural Language Queries"},{"task_slug":"speech-enhancement","task_name":"Speech Enhancement"},{"task_slug":"zero-shot-generalization","task_name":"Zero-shot Generalization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2308.05037","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.05037"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/audio-agi/audiosep","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":5,"ran_draft_wrong":1,"ran_violates":2,"unverified":1},"by_repo_kind":{"official":{"samples":9,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"3d4fcbbce6ace804","entry":"act","repo":"audio-agi/audiosep","repo_kind":"official","path":"models/base.py","file_url":"https://github.com/audio-agi/audiosep/blob/HEAD/models/base.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3d4fcbbce6ace804"}},{"code_sha256_prefix":"720ba14465467e12","entry":"create_logging","repo":"audio-agi/audiosep","repo_kind":"official","path":"utils.py","file_url":"https://github.com/audio-agi/audiosep/blob/HEAD/utils.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"720ba14465467e12"}},{"code_sha256_prefix":"ffa5754c94663d62","entry":"float32_to_int16","repo":"audio-agi/audiosep","repo_kind":"official","path":"utils.py","file_url":"https://github.com/audio-agi/audiosep/blob/HEAD/utils.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ffa5754c94663d62"}},{"code_sha256_prefix":"de10184f4a653fce","entry":"get_film_meta","repo":"audio-agi/audiosep","repo_kind":"official","path":"models/resunet.py","file_url":"https://github.com/audio-agi/audiosep/blob/HEAD/models/resunet.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"de10184f4a653fce"}},{"code_sha256_prefix":"619a481adfa3ce11","entry":"get_loss_function","repo":"audio-agi/audiosep","repo_kind":"official","path":"losses.py","file_url":"https://github.com/audio-agi/audiosep/blob/HEAD/losses.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"619a481adfa3ce11"}},{"code_sha256_prefix":"cfd47b51fefe3711","entry":"int16_to_float32","repo":"audio-agi/audiosep","repo_kind":"official","path":"utils.py","file_url":"https://github.com/audio-agi/audiosep/blob/HEAD/utils.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cfd47b51fefe3711"}},{"code_sha256_prefix":"9d43ef1e369fdd71","entry":"l1","repo":"audio-agi/audiosep","repo_kind":"official","path":"losses.py","file_url":"https://github.com/audio-agi/audiosep/blob/HEAD/losses.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9d43ef1e369fdd71"}},{"code_sha256_prefix":"5073e2ca7b39575f","entry":"l1_wav","repo":"audio-agi/audiosep","repo_kind":"official","path":"losses.py","file_url":"https://github.com/audio-agi/audiosep/blob/HEAD/losses.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5073e2ca7b39575f"}},{"code_sha256_prefix":"856732fbad089eeb","entry":"get_model_class","repo":"audio-agi/audiosep","repo_kind":"official","path":"models/audiosep.py","file_url":"https://github.com/audio-agi/audiosep/blob/HEAD/models/audiosep.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"856732fbad089eeb"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}