{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/openvoice-versatile-instant-voice-cloning","title":"OpenVoice: Versatile Instant Voice Cloning","arxiv_id":"2312.01479","date":"2023-12-03","proceeding":null,"authors":["Zengyi Qin","Wenliang Zhao","Xumin Yu","Xin Sun"],"abstract":"We introduce OpenVoice, a versatile voice cloning approach that requires only a short audio clip from the reference speaker to replicate their voice and generate speech in multiple languages. OpenVoice represents a significant advancement in addressing the following open challenges in the field: 1) Flexible Voice Style Control. OpenVoice enables granular control over voice styles, including emotion, accent, rhythm, pauses, and intonation, in addition to replicating the tone color of the reference speaker. The voice styles are not directly copied from and constrained by the style of the reference speaker. Previous approaches lacked the ability to flexibly manipulate voice styles after cloning. 2) Zero-Shot Cross-Lingual Voice Cloning. OpenVoice achieves zero-shot cross-lingual voice cloning for languages not included in the massive-speaker training set. Unlike previous approaches, which typically require extensive massive-speaker multi-lingual (MSML) dataset for all languages, OpenVoice can clone voices into a new language without any massive-speaker training data for that language. OpenVoice is also computationally efficient, costing tens of times less than commercially available APIs that offer even inferior performance. To foster further research in the field, we have made the source code and trained model publicly accessible. We also provide qualitative results in our demo website. OpenVoice has been used by more than 2M users worldwide as the voice engine of MyShell.ai","url_abs":"https://arxiv.org/abs/2312.01479v6","url_pdf":"https://arxiv.org/pdf/2312.01479v6.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"openvoice-versatile-instant-voice-cloning","repo_url":"https://github.com/myshell-ai/openvoice","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"rhythm","task_name":"Rhythm"},{"task_slug":"voice-cloning","task_name":"Voice Cloning"}],"methods":[{"method_slug":"clip","method_name":"CLIP"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2312.01479","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.01479"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/myshell-ai/openvoice","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_fixture":1,"ran_honours":1,"ran":1,"unverified":12},"by_repo_kind":{"named_in_paper":{"samples":15,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"b38f9c28c5117398","entry":"fused_add_tanh_sigmoid_multiply","repo":"myshell-ai/openvoice","repo_kind":"named_in_paper","path":"openvoice/attentions.py","file_url":"https://github.com/myshell-ai/openvoice/blob/HEAD/openvoice/attentions.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b38f9c28c5117398"}},{"code_sha256_prefix":"a26f85d7c72ef39a","entry":"get_padding","repo":"myshell-ai/openvoice","repo_kind":"named_in_paper","path":"openvoice/commons.py","file_url":"https://github.com/myshell-ai/openvoice/blob/HEAD/openvoice/commons.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a26f85d7c72ef39a"}},{"code_sha256_prefix":"70a7614b3b3eb41f","entry":"unconstrained_rational_quadratic_spline","repo":"myshell-ai/openvoice","repo_kind":"named_in_paper","path":"openvoice/transforms.py","file_url":"https://github.com/myshell-ai/openvoice/blob/HEAD/openvoice/transforms.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"70a7614b3b3eb41f"}},{"code_sha256_prefix":"edfa904ec3cd9c56","entry":"bits_to_string","repo":"myshell-ai/openvoice","repo_kind":"named_in_paper","path":"openvoice/utils.py","file_url":"https://github.com/myshell-ai/openvoice/blob/HEAD/openvoice/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"edfa904ec3cd9c56"}},{"code_sha256_prefix":"ca53d51dae7325e7","entry":"collapse_whitespace","repo":"myshell-ai/openvoice","repo_kind":"named_in_paper","path":"openvoice/text/english.py","file_url":"https://github.com/myshell-ai/openvoice/blob/HEAD/openvoice/text/english.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ca53d51dae7325e7"}},{"code_sha256_prefix":"feec90b898e6e43b","entry":"convert_pad_shape","repo":"myshell-ai/openvoice","repo_kind":"named_in_paper","path":"openvoice/commons.py","file_url":"https://github.com/myshell-ai/openvoice/blob/HEAD/openvoice/commons.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"feec90b898e6e43b"}},{"code_sha256_prefix":"262807b9eb9706a5","entry":"dynamic_range_compression_torch","repo":"myshell-ai/openvoice","repo_kind":"named_in_paper","path":"openvoice/mel_processing.py","file_url":"https://github.com/myshell-ai/openvoice/blob/HEAD/openvoice/mel_processing.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"262807b9eb9706a5"}},{"code_sha256_prefix":"78c12e563d5af53d","entry":"dynamic_range_decompression_torch","repo":"myshell-ai/openvoice","repo_kind":"named_in_paper","path":"openvoice/mel_processing.py","file_url":"https://github.com/myshell-ai/openvoice/blob/HEAD/openvoice/mel_processing.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"78c12e563d5af53d"}},{"code_sha256_prefix":"cea1f83e9db28ed8","entry":"expand_abbreviations","repo":"myshell-ai/openvoice","repo_kind":"named_in_paper","path":"openvoice/text/english.py","file_url":"https://github.com/myshell-ai/openvoice/blob/HEAD/openvoice/text/english.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cea1f83e9db28ed8"}},{"code_sha256_prefix":"8864cfac554d8105","entry":"get_hparams_from_file","repo":"myshell-ai/openvoice","repo_kind":"named_in_paper","path":"openvoice/utils.py","file_url":"https://github.com/myshell-ai/openvoice/blob/HEAD/openvoice/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8864cfac554d8105"}},{"code_sha256_prefix":"c01a3efb274ff778","entry":"intersperse","repo":"myshell-ai/openvoice","repo_kind":"named_in_paper","path":"openvoice/commons.py","file_url":"https://github.com/myshell-ai/openvoice/blob/HEAD/openvoice/commons.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c01a3efb274ff778"}},{"code_sha256_prefix":"ba6cafc4d658c14f","entry":"piecewise_rational_quadratic_transform","repo":"myshell-ai/openvoice","repo_kind":"named_in_paper","path":"openvoice/transforms.py","file_url":"https://github.com/myshell-ai/openvoice/blob/HEAD/openvoice/transforms.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ba6cafc4d658c14f"}},{"code_sha256_prefix":"e85d747983c4f5d8","entry":"searchsorted","repo":"myshell-ai/openvoice","repo_kind":"named_in_paper","path":"openvoice/transforms.py","file_url":"https://github.com/myshell-ai/openvoice/blob/HEAD/openvoice/transforms.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e85d747983c4f5d8"}},{"code_sha256_prefix":"26b87488d429896a","entry":"spectral_normalize_torch","repo":"myshell-ai/openvoice","repo_kind":"named_in_paper","path":"openvoice/mel_processing.py","file_url":"https://github.com/myshell-ai/openvoice/blob/HEAD/openvoice/mel_processing.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"26b87488d429896a"}},{"code_sha256_prefix":"8f51f7777876e56d","entry":"string_to_bits","repo":"myshell-ai/openvoice","repo_kind":"named_in_paper","path":"openvoice/utils.py","file_url":"https://github.com/myshell-ai/openvoice/blob/HEAD/openvoice/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8f51f7777876e56d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}