{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dynamic-programming-in-rank-space-scaling-1","title":"Dynamic Programming in Rank Space: Scaling Structured Inference with Low-Rank HMMs and PCFGs","arxiv_id":"2205.00484","date":"2022-05-01","proceeding":"NAACL 2022 7","authors":["Songlin Yang","Wei Liu","Kewei Tu"],"abstract":"Hidden Markov Models (HMMs) and Probabilistic Context-Free Grammars (PCFGs) are widely used structured models, both of which can be represented as factor graph grammars (FGGs), a powerful formalism capable of describing a wide range of models. Recent research found it beneficial to use large state spaces for HMMs and PCFGs. However, inference with large state spaces is computationally demanding, especially for PCFGs. To tackle this challenge, we leverage tensor rank decomposition (aka.\\ CPD) to decrease inference computational complexities for a subset of FGGs subsuming HMMs and PCFGs. We apply CPD on the factors of an FGG and then construct a new FGG defined in the rank space. Inference with the new FGG produces the same result but has a lower time complexity when the rank size is smaller than the state size. We conduct experiments on HMM language modeling and unsupervised PCFG parsing, showing better performance than previous work. Our code is publicly available at \\url{https://github.com/VPeterV/RankSpace-Models}.","url_abs":"https://arxiv.org/abs/2205.00484v1","url_pdf":"https://arxiv.org/pdf/2205.00484v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dynamic-programming-in-rank-space-scaling-1","repo_url":"https://github.com/sustcsonglin/TN-PCFG","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"dynamic-programming-in-rank-space-scaling-1","repo_url":"https://github.com/vpeterv/rankspace-models","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"constituency-grammar-induction","task_name":"Constituency Grammar Induction"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/constituency-grammar-induction-on-ptb","task":"Constituency Grammar Induction","dataset":"PTB Diagnostic ECG Database","model":"DP in rank space","rank_in_archive_order":7,"of":24,"metrics":{"Mean F1 (WSJ)":"64.1"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2205.00484","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}