{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-hierarchical-latent-structure-for","title":"A Hierarchical Latent Structure for Variational Conversation Modeling","arxiv_id":"1804.03424","date":"2018-04-10","proceeding":"NAACL 2018 6","authors":["Yookoon Park","Jaemin Cho","Gunhee Kim"],"abstract":"Variational autoencoders (VAE) combined with hierarchical RNNs have emerged\nas a powerful framework for conversation modeling. However, they suffer from\nthe notorious degeneration problem, where the decoders learn to ignore latent\nvariables and reduce to vanilla RNNs. We empirically show that this degeneracy\noccurs mostly due to two reasons. First, the expressive power of hierarchical\nRNN decoders is often high enough to model the data using only its decoding\ndistributions without relying on the latent variables. Second, the conditional\nVAE structure whose generation process is conditioned on a context, makes the\nrange of training targets very sparse; that is, the RNN decoders can easily\noverfit to the training data ignoring the latent variables. To solve the\ndegeneration problem, we propose a novel model named Variational Hierarchical\nConversation RNNs (VHCR), involving two key ideas of (1) using a hierarchical\nstructure of latent variables, and (2) exploiting an utterance drop\nregularization. With evaluations on two datasets of Cornell Movie Dialog and\nUbuntu Dialog Corpus, we show that our VHCR successfully utilizes latent\nvariables and outperforms state-of-the-art models for conversation generation.\nMoreover, it can perform several new utterance control tasks, thanks to its\nhierarchical latent structure.","url_abs":"http://arxiv.org/abs/1804.03424v2","url_pdf":"http://arxiv.org/pdf/1804.03424v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-hierarchical-latent-structure-for","repo_url":"https://github.com/Tanasho0928/chat-oriented","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"a-hierarchical-latent-structure-for","repo_url":"https://github.com/Tanasho0928/ncm","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"a-hierarchical-latent-structure-for","repo_url":"https://github.com/ctr4si/A-Hierarchical-Latent-Structure-for-Variational-Conversation-Modeling","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"a-hierarchical-latent-structure-for","repo_url":"https://github.com/natashamjaques/neural_chat","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1804.03424","atlas_url":"https://app.syntology.ai/?focus=1804.03424","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}