{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/kermit-complementing-transformer","title":"KERMIT: Complementing Transformer Architectures with Encoders of Explicit Syntactic Interpretations","arxiv_id":null,"date":"2020-11-01","proceeding":"EMNLP 2020 11","authors":["Fabio Massimo Zanzotto","Andrea Santilli","Leonardo Ranaldi","Dario Onorati","Pierfrancesco Tommasino","Francesca Fallucchi"],"abstract":"Syntactic parsers have dominated natural language understanding for decades. Yet, their syntactic interpretations are losing centrality in downstream tasks due to the success of large-scale textual representation learners. In this paper, we propose KERMIT (Kernel-inspired Encoder with Recursive Mechanism for Interpretable Trees) to embed symbolic syntactic parse trees into artificial neural networks and to visualize how syntax is used in inference. We experimented with KERMIT paired with two state-of-the-art transformer-based universal sentence encoders (BERT and XLNet) and we showed that KERMIT can indeed boost their performance by effectively embedding human-coded universal syntactic representations in neural networks","url_abs":"https://aclanthology.org/2020.emnlp-main.18","url_pdf":"https://aclanthology.org/2020.emnlp-main.18.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"kermit-complementing-transformer","repo_url":"https://github.com/ART-Group-it/KERMIT","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"syntax-representation","task_name":"Syntax Representation"},{"task_slug":"text-classification","task_name":"Text Classification"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"bert","method_name":"BERT"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"syntax-heat-parse-tree","method_name":"Syntax Heat Parse Tree"},{"method_slug":"transformer","method_name":"Transformer"},{"method_slug":"xlnet","method_name":"XLNet"}],"datasets_introduced":[],"methods_introduced":[{"slug":"syntax-heat-parse-tree","name":"Syntax Heat Parse Tree","full_name":"Syntax Heat Parse Tree"}],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}