{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/rmssinger-realistic-music-score-based-singing","title":"RMSSinger: Realistic-Music-Score based Singing Voice Synthesis","arxiv_id":"2305.10686","date":"2023-05-18","proceeding":null,"authors":["Jinzheng He","Jinglin Liu","Zhenhui Ye","Rongjie Huang","Chenye Cui","Huadai Liu","Zhou Zhao"],"abstract":"We are interested in a challenging task, Realistic-Music-Score based Singing Voice Synthesis (RMS-SVS). RMS-SVS aims to generate high-quality singing voices given realistic music scores with different note types (grace, slur, rest, etc.). Though significant progress has been achieved, recent singing voice synthesis (SVS) methods are limited to fine-grained music scores, which require a complicated data collection pipeline with time-consuming manual annotation to align music notes with phonemes. Furthermore, these manual annotation destroys the regularity of note durations in music scores, making fine-grained music scores inconvenient for composing. To tackle these challenges, we propose RMSSinger, the first RMS-SVS method, which takes realistic music scores as input, eliminating most of the tedious manual annotation and avoiding the aforementioned inconvenience. Note that music scores are based on words rather than phonemes, in RMSSinger, we introduce word-level modeling to avoid the time-consuming phoneme duration annotation and the complicated phoneme-level mel-note alignment. Furthermore, we propose the first diffusion-based pitch modeling method, which ameliorates the naturalness of existing pitch-modeling methods. To achieve these, we collect a new dataset containing realistic music scores and singing voices according to these realistic music scores from professional singers. Extensive experiments on the dataset demonstrate the effectiveness of our methods. Audio samples are available at https://rmssinger.github.io/.","url_abs":"https://arxiv.org/abs/2305.10686v1","url_pdf":"https://arxiv.org/pdf/2305.10686v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"rmssinger-realistic-music-score-based-singing","repo_url":"https://github.com/tonnetonne814/SiFi-VITS2-44100-Ja","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"singing-voice-synthesis","task_name":"Singing Voice Synthesis"}],"methods":[{"method_slug":"align","method_name":"ALIGN"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.10686","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.10686"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tonnetonne814/SiFi-VITS2-44100-Ja","reach":null}],"summary":{"ran_fixture":2},"by_repo_kind":{"listed":{"samples":2,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d3a9c00767735ee2","entry":"generate_graph","repo":"tonnetonne814/SiFi-VITS2-44100-Ja","repo_kind":"listed","path":"train_SVS_synthesizer.py","file_url":"https://github.com/tonnetonne814/SiFi-VITS2-44100-Ja/blob/HEAD/train_SVS_synthesizer.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d3a9c00767735ee2"}},{"code_sha256_prefix":"33374877ef882af3","entry":"moving_average","repo":"tonnetonne814/SiFi-VITS2-44100-Ja","repo_kind":"listed","path":"train_SVS_synthesizer.py","file_url":"https://github.com/tonnetonne814/SiFi-VITS2-44100-Ja/blob/HEAD/train_SVS_synthesizer.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"33374877ef882af3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}