{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/sonicsim-a-customizable-simulation-platform","title":"SonicSim: A customizable simulation platform for speech processing in moving sound source scenarios","arxiv_id":"2410.01481","date":"2024-10-02","proceeding":null,"authors":["Kai Li","Wendi Sang","Chang Zeng","Runxuan Yang","Guo Chen","Xiaolin Hu"],"abstract":"Systematic evaluation of speech separation and enhancement models under moving sound source conditions requires extensive and diverse data. However, real-world datasets often lack sufficient data for training and evaluation, and synthetic datasets, while larger, lack acoustic realism. Consequently, neither effectively meets practical needs. To address this issue, we introduce SonicSim, a synthetic toolkit based on the embodied AI simulation platform Habitat-sim, designed to generate highly customizable data for moving sound sources. SonicSim supports multi-level adjustments, including scene-level, microphone-level, and source-level adjustments, enabling the creation of more diverse synthetic data. Leveraging SonicSim, we constructed a benchmark dataset called SonicSet, utilizing LibriSpeech, Freesound Dataset 50k (FSD50K), Free Music Archive (FMA), and 90 scenes from Matterport3D to evaluate speech separation and enhancement models. Additionally, to investigate the differences between synthetic and real-world data, we selected 5 hours of raw, non-reverberant data from the SonicSet validation set and recorded a real-world speech separation dataset, providing a reference for comparing SonicSet with other synthetic datasets. For speech enhancement, we utilized the real-world dataset RealMAN to validate the acoustic gap between SonicSet and existing synthetic datasets. The results indicate that models trained on SonicSet generalize better to real-world scenarios compared to other synthetic datasets. The code is publicly available at https://cslikai.cn/SonicSim/.","url_abs":"https://arxiv.org/abs/2410.01481v2","url_pdf":"https://arxiv.org/pdf/2410.01481v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"sonicsim-a-customizable-simulation-platform","repo_url":"https://github.com/jusperlee/sonicsim","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"CC-BY-SA-4.0"}},{"paper_slug":"sonicsim-a-customizable-simulation-platform","repo_url":"https://github.com/jusperlee/librispace","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"CC-BY-SA-4.0"}}],"tasks":[{"task_slug":"speech-enhancement","task_name":"Speech Enhancement"},{"task_slug":"speech-separation","task_name":"Speech Separation"}],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.01481","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.01481"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jusperlee/librispace","reach":{"status":"ok","spdx":"CC-BY-SA-4.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jusperlee/sonicsim","reach":{"status":"ok","spdx":"CC-BY-SA-4.0"}}],"summary":{"ran":7,"ran_draft_wrong":1,"unverified":3},"by_repo_kind":{"official":{"samples":11,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":11,"samples":[{"code_sha256_prefix":"ac140dd3edbc9913","entry":"clip_all","repo":"jusperlee/sonicsim","repo_kind":"official","path":"SonicSim-SonicSet/SonicSim_rir.py","file_url":"https://github.com/jusperlee/sonicsim/blob/HEAD/SonicSim-SonicSet/SonicSim_rir.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"ac140dd3edbc9913"}},{"code_sha256_prefix":"d8e74e167e0c6d61","entry":"convolve_fixed_receiver","repo":"jusperlee/sonicsim","repo_kind":"official","path":"SonicSim-SonicSet/SonicSim_moving.py","file_url":"https://github.com/jusperlee/sonicsim/blob/HEAD/SonicSim-SonicSet/SonicSim_moving.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"d8e74e167e0c6d61"}},{"code_sha256_prefix":"8135237c1289f718","entry":"fft_conv","repo":"jusperlee/sonicsim","repo_kind":"official","path":"SonicSim-SonicSet/SonicSim_audio.py","file_url":"https://github.com/jusperlee/sonicsim/blob/HEAD/SonicSim-SonicSet/SonicSim_audio.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"8135237c1289f718"}},{"code_sha256_prefix":"a02604f27b55c6f8","entry":"find_matching_indices","repo":"jusperlee/sonicsim","repo_kind":"official","path":"SonicSim-SonicSet/tool_utils.py","file_url":"https://github.com/jusperlee/sonicsim/blob/HEAD/SonicSim-SonicSet/tool_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"a02604f27b55c6f8"}},{"code_sha256_prefix":"8ec93e1ad184931b","entry":"load_transcripts","repo":"jusperlee/sonicsim","repo_kind":"official","path":"SonicSim-SonicSet/tool_utils.py","file_url":"https://github.com/jusperlee/sonicsim/blob/HEAD/SonicSim-SonicSet/tool_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"8ec93e1ad184931b"}},{"code_sha256_prefix":"40436222d0daa829","entry":"normalize","repo":"jusperlee/sonicsim","repo_kind":"official","path":"SonicSim-SonicSet/SonicSim_audio.py","file_url":"https://github.com/jusperlee/sonicsim/blob/HEAD/SonicSim-SonicSet/SonicSim_audio.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"40436222d0daa829"}},{"code_sha256_prefix":"e2301847cc369177","entry":"removing_exist_speaker","repo":"JusperLee/SonicSim","repo_kind":"official","path":"SonicSim-SonicSet/SonicSet.py","file_url":"https://github.com/JusperLee/SonicSim/blob/HEAD/SonicSim-SonicSet/SonicSet.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"e2301847cc369177"}},{"code_sha256_prefix":"564547fbd7f8e897","entry":"setup_dynamic_interp","repo":"jusperlee/sonicsim","repo_kind":"official","path":"SonicSim-SonicSet/SonicSim_moving.py","file_url":"https://github.com/jusperlee/sonicsim/blob/HEAD/SonicSim-SonicSet/SonicSim_moving.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"564547fbd7f8e897"}},{"code_sha256_prefix":"28b3af59d586d5ee","entry":"GaGNet_wav","repo":"jusperlee/sonicsim","repo_kind":"official","path":"enhancement/audio_test_music.py","file_url":"https://github.com/jusperlee/sonicsim/blob/HEAD/enhancement/audio_test_music.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"28b3af59d586d5ee"}},{"code_sha256_prefix":"8ce28837bd79b8ac","entry":"TaylorWav","repo":"jusperlee/sonicsim","repo_kind":"official","path":"enhancement/audio_test_music.py","file_url":"https://github.com/jusperlee/sonicsim/blob/HEAD/enhancement/audio_test_music.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"8ce28837bd79b8ac"}},{"code_sha256_prefix":"6edbe9425e6ed403","entry":"convolve_moving_receiver","repo":"jusperlee/sonicsim","repo_kind":"official","path":"SonicSim-SonicSet/SonicSim_moving.py","file_url":"https://github.com/jusperlee/sonicsim/blob/HEAD/SonicSim-SonicSet/SonicSim_moving.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC-BY-SA-4.0","inline_ok":false,"mcp_get_code":{"code_sha256":"6edbe9425e6ed403"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}