{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/recurrent-neural-network-based-semantic","title":"Recurrent Neural Network-Based Semantic Variational Autoencoder for Sequence-to-Sequence Learning","arxiv_id":"1802.03238","date":"2018-02-09","proceeding":null,"authors":["Myeongjun Jang","Seungwan Seo","Pilsung Kang"],"abstract":"Sequence-to-sequence (Seq2seq) models have played an important role in the\nrecent success of various natural language processing methods, such as machine\ntranslation, text summarization, and speech recognition. However, current\nSeq2seq models have trouble preserving global latent information from a long\nsequence of words. Variational autoencoder (VAE) alleviates this problem by\nlearning a continuous semantic space of the input sentence. However, it does\nnot solve the problem completely. In this paper, we propose a new recurrent\nneural network (RNN)-based Seq2seq model, RNN semantic variational autoencoder\n(RNN--SVAE), to better capture the global latent information of a sequence of\nwords. To reflect the meaning of words in a sentence properly, without regard\nto its position within the sentence, we construct a document information vector\nusing the attention information between the final state of the encoder and\nevery prior hidden state. Then, the mean and standard deviation of the\ncontinuous semantic space are learned by using this vector to take advantage of\nthe variational method. By using the document information vector to find the\nsemantic space of the sentence, it becomes possible to better capture the\nglobal latent feature of the sentence. Experimental results of three natural\nlanguage tasks (i.e., language modeling, missing word imputation, paraphrase\nidentification) confirm that the proposed RNN--SVAE yields higher performance\nthan two benchmark models.","url_abs":"http://arxiv.org/abs/1802.03238v2","url_pdf":"http://arxiv.org/pdf/1802.03238v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"recurrent-neural-network-based-semantic","repo_url":"https://github.com/MJ-Jang/RNN_SVAE","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"imputation","task_name":"Imputation"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"paraphrase-identification","task_name":"Paraphrase Identification"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"speech-recognition","task_name":"Speech Recognition"},{"task_slug":"text-summarization","task_name":"Text Summarization"},{"task_slug":"speech-recognition-1","task_name":"speech-recognition"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"seq2seq","method_name":"Seq2Seq"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}