{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/spherical-latent-spaces-for-stable","title":"Spherical Latent Spaces for Stable Variational Autoencoders","arxiv_id":"1808.10805","date":"2018-08-31","proceeding":"EMNLP 2018 10","authors":["Jiacheng Xu","Greg Durrett"],"abstract":"A hallmark of variational autoencoders (VAEs) for text processing is their\ncombination of powerful encoder-decoder models, such as LSTMs, with simple\nlatent distributions, typically multivariate Gaussians. These models pose a\ndifficult optimization problem: there is an especially bad local optimum where\nthe variational posterior always equals the prior and the model does not use\nthe latent variable at all, a kind of \"collapse\" which is encouraged by the KL\ndivergence term of the objective. In this work, we experiment with another\nchoice of latent distribution, namely the von Mises-Fisher (vMF) distribution,\nwhich places mass on the surface of the unit hypersphere. With this choice of\nprior and posterior, the KL divergence term now only depends on the variance of\nthe vMF distribution, giving us the ability to treat it as a fixed\nhyperparameter. We show that doing so not only averts the KL collapse, but\nconsistently gives better likelihoods than Gaussians across a range of modeling\nconditions, including recurrent language modeling and bag-of-words document\nmodeling. An analysis of the properties of our vMF representations shows that\nthey learn richer and more nuanced structures in their latent representations\nthan their Gaussian counterparts.","url_abs":"http://arxiv.org/abs/1808.10805v2","url_pdf":"http://arxiv.org/pdf/1808.10805v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"spherical-latent-spaces-for-stable","repo_url":"https://github.com/jiacheng-xu/vmf_vae_nlp","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"topic-models","task_name":"Topic Models"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/topic-models-on-20newsgroups","task":"Topic Models","dataset":"20NewsGroups","model":"vNVDM","rank_in_archive_order":3,"of":6,"metrics":{"C_v":"0.52"},"uses_additional_data":false},{"leaderboard":"/sota/topic-models-on-ag-news","task":"Topic Models","dataset":"AG News","model":"vNVDM","rank_in_archive_order":2,"of":6,"metrics":{"C_v":"0.44","NPMI":"0.028"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1808.10805","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1808.10805"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jiacheng-xu/vmf_vae_nlp","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":6},"by_repo_kind":{"official":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ee445ccbb171f50b","entry":"KL_davidson","repo":"jiacheng-xu/vmf_vae_nlp","repo_kind":"official","path":"NVLL/distribution/vmf_hypvae.py","file_url":"https://github.com/jiacheng-xu/vmf_vae_nlp/blob/HEAD/NVLL/distribution/vmf_hypvae.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ee445ccbb171f50b"}},{"code_sha256_prefix":"bc6de48d07ae7aee","entry":"KL_guu","repo":"jiacheng-xu/vmf_vae_nlp","repo_kind":"official","path":"NVLL/distribution/vmf_hypvae.py","file_url":"https://github.com/jiacheng-xu/vmf_vae_nlp/blob/HEAD/NVLL/distribution/vmf_hypvae.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bc6de48d07ae7aee"}},{"code_sha256_prefix":"6684731d8353c4c7","entry":"drawSphere","repo":"jiacheng-xu/vmf_vae_nlp","repo_kind":"official","path":"NVLL/visual/draw_vmf_ball.py","file_url":"https://github.com/jiacheng-xu/vmf_vae_nlp/blob/HEAD/NVLL/visual/draw_vmf_ball.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6684731d8353c4c7"}},{"code_sha256_prefix":"db76e6258aa90f1a","entry":"draw_ball","repo":"jiacheng-xu/vmf_vae_nlp","repo_kind":"official","path":"NVLL/visual/draw_vmf_ball.py","file_url":"https://github.com/jiacheng-xu/vmf_vae_nlp/blob/HEAD/NVLL/visual/draw_vmf_ball.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"db76e6258aa90f1a"}},{"code_sha256_prefix":"e8b09bdadb7dfa33","entry":"make_rand_vector","repo":"jiacheng-xu/vmf_vae_nlp","repo_kind":"official","path":"NVLL/visual/draw_vmf_ball.py","file_url":"https://github.com/jiacheng-xu/vmf_vae_nlp/blob/HEAD/NVLL/visual/draw_vmf_ball.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e8b09bdadb7dfa33"}},{"code_sha256_prefix":"de66575f575699c3","entry":"mod_Bessel_first_kind","repo":"jiacheng-xu/vmf_vae_nlp","repo_kind":"official","path":"NVLL/distribution/archived_vmf.py","file_url":"https://github.com/jiacheng-xu/vmf_vae_nlp/blob/HEAD/NVLL/distribution/archived_vmf.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"de66575f575699c3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}