{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unsupervised-learning-of-syntactic-structure","title":"Unsupervised Learning of Syntactic Structure with Invertible Neural Projections","arxiv_id":"1808.09111","date":"2018-08-28","proceeding":"EMNLP 2018 10","authors":["Junxian He","Graham Neubig","Taylor Berg-Kirkpatrick"],"abstract":"Unsupervised learning of syntactic structure is typically performed using\ngenerative models with discrete latent variables and multinomial parameters. In\nmost cases, these models have not leveraged continuous word representations. In\nthis work, we propose a novel generative model that jointly learns discrete\nsyntactic structure and continuous word representations in an unsupervised\nfashion by cascading an invertible neural network with a structured generative\nprior. We show that the invertibility condition allows for efficient exact\ninference and marginal likelihood computation in our model so long as the prior\nis well-behaved. In experiments we instantiate our approach with both Markov\nand tree-structured priors, evaluating on two tasks: part-of-speech (POS)\ninduction, and unsupervised dependency parsing without gold POS annotation. On\nthe Penn Treebank, our Markov-structured model surpasses state-of-the-art\nresults on POS induction. Similarly, we find that our tree-structured model\nachieves state-of-the-art performance on unsupervised dependency parsing for\nthe difficult training condition where neither gold POS annotation nor\npunctuation-based constraints are available.","url_abs":"http://arxiv.org/abs/1808.09111v1","url_pdf":"http://arxiv.org/pdf/1808.09111v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unsupervised-learning-of-syntactic-structure","repo_url":"https://github.com/jxhe/struct-learning-with-flow","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"constituency-grammar-induction","task_name":"Constituency Grammar Induction"},{"task_slug":"dependency-parsing","task_name":"Dependency Parsing"},{"task_slug":"pos","task_name":"POS"},{"task_slug":"unsupervised-dependency-parsing","task_name":"Unsupervised Dependency Parsing"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/constituency-grammar-induction-on-ptb","task":"Constituency Grammar Induction","dataset":"PTB Diagnostic ECG Database","model":"DMV + invertible projector","rank_in_archive_order":20,"of":24,"metrics":{"Mean F1 (WSJ)":"47.9","Mean F1 (WSJ10)":"60.2"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1808.09111","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}