{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/binauralgrad-a-two-stage-conditional","title":"BinauralGrad: A Two-Stage Conditional Diffusion Probabilistic Model for Binaural Audio Synthesis","arxiv_id":"2205.14807","date":"2022-05-30","proceeding":null,"authors":["Yichong Leng","Zehua Chen","Junliang Guo","Haohe Liu","Jiawei Chen","Xu Tan","Danilo Mandic","Lei He","Xiang-Yang Li","Tao Qin","Sheng Zhao","Tie-Yan Liu"],"abstract":"Binaural audio plays a significant role in constructing immersive augmented and virtual realities. As it is expensive to record binaural audio from the real world, synthesizing them from mono audio has attracted increasing attention. This synthesis process involves not only the basic physical warping of the mono audio, but also room reverberations and head/ear related filtrations, which, however, are difficult to accurately simulate in traditional digital signal processing. In this paper, we formulate the synthesis process from a different perspective by decomposing the binaural audio into a common part that shared by the left and right channels as well as a specific part that differs in each channel. Accordingly, we propose BinauralGrad, a novel two-stage framework equipped with diffusion models to synthesize them respectively. Specifically, in the first stage, the common information of the binaural audio is generated with a single-channel diffusion model conditioned on the mono audio, based on which the binaural audio is generated by a two-channel diffusion model in the second stage. Combining this novel perspective of two-stage synthesis with advanced generative models (i.e., the diffusion models),the proposed BinauralGrad is able to generate accurate and high-fidelity binaural audio samples. Experiment results show that on a benchmark dataset, BinauralGrad outperforms the existing baselines by a large margin in terms of both object and subject evaluation metrics (Wave L2: 0.128 vs. 0.157, MOS: 3.80 vs. 3.61). The generated audio samples (https://speechresearch.github.io/binauralgrad) and code (https://github.com/microsoft/NeuralSpeech/tree/master/BinauralGrad) are available online.","url_abs":"https://arxiv.org/abs/2205.14807v2","url_pdf":"https://arxiv.org/pdf/2205.14807v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"binauralgrad-a-two-stage-conditional","repo_url":"https://github.com/microsoft/NeuralSpeech","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"audio-synthesis","task_name":"Audio Synthesis"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2205.14807","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2205.14807"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/microsoft/NeuralSpeech","reach":null}],"summary":{"ran":1,"ran_fixture":1,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f2e63d7f15a1abb8","entry":"BinauralPreNet","repo":"microsoft/NeuralSpeech","repo_kind":"official","path":"BinauralGrad/src/binauralgrad/model.py","file_url":"https://github.com/microsoft/NeuralSpeech/blob/HEAD/BinauralGrad/src/binauralgrad/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f2e63d7f15a1abb8"}},{"code_sha256_prefix":"ea4b5e975b1e2ba8","entry":"silu","repo":"microsoft/neuralspeech","repo_kind":"official","path":"BinauralGrad/src/binauralgrad/model.py","file_url":"https://github.com/microsoft/neuralspeech/blob/HEAD/BinauralGrad/src/binauralgrad/model.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ea4b5e975b1e2ba8"}},{"code_sha256_prefix":"603764b10da2797f","entry":"BinauralGrad","repo":"microsoft/NeuralSpeech","repo_kind":"official","path":"BinauralGrad/src/binauralgrad/model.py","file_url":"https://github.com/microsoft/NeuralSpeech/blob/HEAD/BinauralGrad/src/binauralgrad/model.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"603764b10da2797f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}