{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-topic-models-going-beyond-svd","title":"Learning Topic Models - Going beyond SVD","arxiv_id":"1204.1956","date":"2012-04-09","proceeding":null,"authors":["Sanjeev Arora","Rong Ge","Ankur Moitra"],"abstract":"Topic Modeling is an approach used for automatic comprehension and\nclassification of data in a variety of settings, and perhaps the canonical\napplication is in uncovering thematic structure in a corpus of documents. A\nnumber of foundational works both in machine learning and in theory have\nsuggested a probabilistic model for documents, whereby documents arise as a\nconvex combination of (i.e. distribution on) a small number of topic vectors,\neach topic vector being a distribution on words (i.e. a vector of\nword-frequencies). Similar models have since been used in a variety of\napplication areas; the Latent Dirichlet Allocation or LDA model of Blei et al.\nis especially popular.\n  Theoretical studies of topic modeling focus on learning the model's\nparameters assuming the data is actually generated from it. Existing approaches\nfor the most part rely on Singular Value Decomposition(SVD), and consequently\nhave one of two limitations: these works need to either assume that each\ndocument contains only one topic, or else can only recover the span of the\ntopic vectors instead of the topic vectors themselves.\n  This paper formally justifies Nonnegative Matrix Factorization(NMF) as a main\ntool in this context, which is an analog of SVD where all vectors are\nnonnegative. Using this tool we give the first polynomial-time algorithm for\nlearning topic models without the above two limitations. The algorithm uses a\nfairly mild assumption about the underlying topic matrix called separability,\nwhich is usually found to hold in real-life data. A compelling feature of our\nalgorithm is that it generalizes to models that incorporate topic-topic\ncorrelations, such as the Correlated Topic Model and the Pachinko Allocation\nModel.\n  We hope that this paper will motivate further theoretical results that use\nNMF as a replacement for SVD - just as NMF has come to replace SVD in many\napplications.","url_abs":"http://arxiv.org/abs/1204.1956v2","url_pdf":"http://arxiv.org/pdf/1204.1956v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-topic-models-going-beyond-svd","repo_url":"https://github.com/moontae/jsmf-raw","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"learning-topic-models-going-beyond-svd","repo_url":"https://github.com/sc782/pyJSMF-RAW","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"topic-models","task_name":"Topic Models"}],"methods":[{"method_slug":"lda","method_name":"LDA"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1204.1956","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}