{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/re-examination-of-the-role-of-latent","title":"Re-examination of the Role of Latent Variables in Sequence Modeling","arxiv_id":"1902.01388","date":"2019-02-04","proceeding":"NeurIPS 2019 12","authors":["Zihang Dai","Guokun Lai","Yiming Yang","Shinjae Yoo"],"abstract":"With latent variables, stochastic recurrent models have achieved state-of-the-art performance in modeling sound-wave sequence. However, opposite results are also observed in other domains, where standard recurrent networks often outperform stochastic models. To better understand this discrepancy, we re-examine the roles of latent variables in stochastic recurrent models for speech density estimation. Our analysis reveals that under the restriction of fully factorized output distribution in previous evaluations, the stochastic models were implicitly leveraging intra-step correlation but the standard recurrent baselines were prohibited to do so, resulting in an unfair comparison. To correct the unfairness, we remove such restriction in our re-examination, where all the models can explicitly leverage intra-step correlation with an auto-regressive structure. Over a diverse set of sequential data, including human speech, MIDI music, handwriting trajectory and frame-permuted speech, our results show that stochastic recurrent models fail to exhibit any practical advantage despite the claimed theoretical superiority. In contrast, standard recurrent models equipped with an auto-regressive output distribution consistently perform better, significantly advancing the state-of-the-art results on three speech datasets.","url_abs":"https://arxiv.org/abs/1902.01388v2","url_pdf":"https://arxiv.org/pdf/1902.01388v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"re-examination-of-the-role-of-latent","repo_url":"https://github.com/zihangdai/reexamine-srnn","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"density-estimation","task_name":"Density Estimation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1902.01388","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1902.01388"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zihangdai/reexamine-srnn","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":1,"unverified":2},"by_repo_kind":{"official":{"samples":3,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"90b79fe15491de83","entry":"is_master","repo":"zihangdai/reexamine-srnn","repo_kind":"official","path":"handwriting/distributed_utils.py","file_url":"https://github.com/zihangdai/reexamine-srnn/blob/HEAD/handwriting/distributed_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"90b79fe15491de83"}},{"code_sha256_prefix":"59755e76d7c3f48b","entry":"all_reduce","repo":"zihangdai/reexamine-srnn","repo_kind":"official","path":"handwriting/distributed_utils.py","file_url":"https://github.com/zihangdai/reexamine-srnn/blob/HEAD/handwriting/distributed_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"59755e76d7c3f48b"}},{"code_sha256_prefix":"74ace08c96374956","entry":"distributed_init","repo":"zihangdai/reexamine-srnn","repo_kind":"official","path":"handwriting/distributed_utils.py","file_url":"https://github.com/zihangdai/reexamine-srnn/blob/HEAD/handwriting/distributed_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"74ace08c96374956"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}