{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/benchmarking-generative-latent-variable","title":"Benchmarking Generative Latent Variable Models for Speech","arxiv_id":"2202.12707","date":"2022-02-22","proceeding":null,"authors":["Jakob D. Havtorn","Lasse Borgholt","Søren Hauberg","Jes Frellsen","Lars Maaløe"],"abstract":"Stochastic latent variable models (LVMs) achieve state-of-the-art performance on natural image generation but are still inferior to deterministic models on speech. In this paper, we develop a speech benchmark of popular temporal LVMs and compare them against state-of-the-art deterministic models. We report the likelihood, which is a much used metric in the image domain, but rarely, or incomparably, reported for speech models. To assess the quality of the learned representations, we also compare their usefulness for phoneme recognition. Finally, we adapt the Clockwork VAE, a state-of-the-art temporal LVM for video generation, to the speech domain. Despite being autoregressive only in latent space, we find that the Clockwork VAE can outperform previous LVMs and reduce the gap to deterministic models by using a hierarchy of latent variables.","url_abs":"https://arxiv.org/abs/2202.12707v2","url_pdf":"https://arxiv.org/pdf/2202.12707v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"benchmarking-generative-latent-variable","repo_url":"https://github.com/jakobhavtorn/benchmarking-lvms","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"phoneme-recognition","task_name":"Phoneme Recognition"},{"task_slug":"video-generation","task_name":"Video Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2202.12707","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2202.12707"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/JakobHavtorn/benchmarking-lvms","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jakobhavtorn/benchmarking-lvms","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":2,"ran_violates":1},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"69763f03a5f7eff7","entry":"add_cuda_device","repo":"jakobhavtorn/benchmarking-lvms","repo_kind":"official","path":"experiments/schedule_experiments.py","file_url":"https://github.com/jakobhavtorn/benchmarking-lvms/blob/HEAD/experiments/schedule_experiments.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"69763f03a5f7eff7"}},{"code_sha256_prefix":"49757e178a320d33","entry":"is_python_command","repo":"jakobhavtorn/benchmarking-lvms","repo_kind":"official","path":"experiments/schedule_experiments.py","file_url":"https://github.com/jakobhavtorn/benchmarking-lvms/blob/HEAD/experiments/schedule_experiments.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"49757e178a320d33"}},{"code_sha256_prefix":"68808a6b2b79bf4f","entry":"make_command_dry","repo":"jakobhavtorn/benchmarking-lvms","repo_kind":"official","path":"experiments/schedule_experiments.py","file_url":"https://github.com/jakobhavtorn/benchmarking-lvms/blob/HEAD/experiments/schedule_experiments.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"68808a6b2b79bf4f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}