{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/exploring-text-queried-sound-event-detection","title":"Exploring Text-Queried Sound Event Detection with Audio Source Separation","arxiv_id":"2409.13292","date":"2024-09-20","proceeding":null,"authors":["Han Yin","Jisheng Bai","Yang Xiao","Hui Wang","Siqi Zheng","Yafeng Chen","Rohan Kumar Das","Chong Deng","Jianfeng Chen"],"abstract":"In sound event detection (SED), overlapping sound events pose a significant challenge, as certain events can be easily masked by background noise or other events, resulting in poor detection performance. To address this issue, we propose the text-queried SED (TQ-SED) framework. Specifically, we first pre-train a language-queried audio source separation (LASS) model to separate the audio tracks corresponding to different events from the input audio. Then, multiple target SED branches are employed to detect individual events. AudioSep is a state-of-the-art LASS model, but has limitations in extracting dynamic audio information because of its pure convolutional structure for separation. To address this, we integrate a dual-path recurrent neural network block into the model. We refer to this structure as AudioSep-DP, which achieves the first place in DCASE 2024 Task 9 on language-queried audio source separation (objective single model track). Experimental results show that TQ-SED can significantly improve the SED performance, with an improvement of 7.22\\% on F1 score over the conventional framework. Additionally, we setup comprehensive experiments to explore the impact of model complexity. The source code and pre-trained model are released at https://github.com/apple-yinhan/TQ-SED.","url_abs":"https://arxiv.org/abs/2409.13292v2","url_pdf":"https://arxiv.org/pdf/2409.13292v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"exploring-text-queried-sound-event-detection","repo_url":"https://github.com/apple-yinhan/tq-sed","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"audio-source-separation","task_name":"Audio Source Separation"},{"task_slug":"event-detection","task_name":"Event Detection"},{"task_slug":"sound-event-detection","task_name":"Sound Event Detection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2409.13292","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2409.13292"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/apple-yinhan/tq-sed","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":8},"by_repo_kind":{"official":{"samples":8,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6dc7818c5777e187","entry":"clip_mse","repo":"apple-yinhan/tq-sed","repo_kind":"official","path":"models/sed.py","file_url":"https://github.com/apple-yinhan/tq-sed/blob/HEAD/models/sed.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6dc7818c5777e187"}},{"code_sha256_prefix":"f68a9049b45e1f45","entry":"event_counts_and_durations","repo":"apple-yinhan/tq-sed","repo_kind":"official","path":"sed_scores_eval/base_modules/ground_truth.py","file_url":"https://github.com/apple-yinhan/tq-sed/blob/HEAD/sed_scores_eval/base_modules/ground_truth.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f68a9049b45e1f45"}},{"code_sha256_prefix":"df0a52d96e1396cc","entry":"fscore_curve_from_intermediate_statistics","repo":"apple-yinhan/tq-sed","repo_kind":"official","path":"sed_scores_eval/base_modules/precision_recall.py","file_url":"https://github.com/apple-yinhan/tq-sed/blob/HEAD/sed_scores_eval/base_modules/precision_recall.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"df0a52d96e1396cc"}},{"code_sha256_prefix":"c8a897b47e7b4f4d","entry":"fscore_from_precision_recall","repo":"apple-yinhan/tq-sed","repo_kind":"official","path":"sed_scores_eval/base_modules/precision_recall.py","file_url":"https://github.com/apple-yinhan/tq-sed/blob/HEAD/sed_scores_eval/base_modules/precision_recall.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c8a897b47e7b4f4d"}},{"code_sha256_prefix":"3f017c80caed3382","entry":"multi_label_to_single_label_ground_truths","repo":"apple-yinhan/tq-sed","repo_kind":"official","path":"sed_scores_eval/base_modules/ground_truth.py","file_url":"https://github.com/apple-yinhan/tq-sed/blob/HEAD/sed_scores_eval/base_modules/ground_truth.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3f017c80caed3382"}},{"code_sha256_prefix":"8314f1aac70f95e9","entry":"onset_deltas","repo":"apple-yinhan/tq-sed","repo_kind":"official","path":"sed_scores_eval/base_modules/detection.py","file_url":"https://github.com/apple-yinhan/tq-sed/blob/HEAD/sed_scores_eval/base_modules/detection.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8314f1aac70f95e9"}},{"code_sha256_prefix":"437c3074d4fee848","entry":"onset_offset_curves","repo":"apple-yinhan/tq-sed","repo_kind":"official","path":"sed_scores_eval/base_modules/detection.py","file_url":"https://github.com/apple-yinhan/tq-sed/blob/HEAD/sed_scores_eval/base_modules/detection.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"437c3074d4fee848"}},{"code_sha256_prefix":"8113a7700e87e50b","entry":"precision_recall_curve_from_intermediate_statistics","repo":"apple-yinhan/tq-sed","repo_kind":"official","path":"sed_scores_eval/base_modules/precision_recall.py","file_url":"https://github.com/apple-yinhan/tq-sed/blob/HEAD/sed_scores_eval/base_modules/precision_recall.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8113a7700e87e50b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}