{"url":"/sota/constituency-parsing-on-penn-treebank","task":{"name":"Constituency Parsing","url":"/task/constituency-parsing","note":null},"dataset":{"name":"Penn Treebank","url":"/dataset/penn-treebank"},"category":"Natural Language Processing","categories":["Natural Language Processing"],"category_note":null,"description":"Constituency parsing aims to extract a constituency-based parse tree from a sentence that\r\nrepresents its syntactic structure according to a [phrase structure grammar](https://en.wikipedia.org/wiki/Phrase_structure_grammar).\r\n\r\nExample:\r\n\r\n                 Sentence (S)\r\n                     |\r\n       +-------------+------------+\r\n       |                          |\r\n     Noun (N)                Verb Phrase (VP)\r\n       |                          |\r\n     John                 +-------+--------+\r\n                          |                |\r\n                        Verb (V)         Noun (N)\r\n                          |                |\r\n                        sees              Bill\r\n\r\n[Recent approaches](https://papers.nips.cc/paper/5635-grammar-as-a-foreign-language.pdf)\r\nconvert the parse tree into a sequence following a depth-first traversal in order to\r\nbe able to apply sequence-to-sequence models to it. The linearized version of the\r\nabove parse tree looks as follows: (S (N) (VP V N)).","description_from":"task","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","rank":"the archive's row order at snapshot; not re-ranked","rows_end_at":"2025-07-28","rows_withheld_as_spam":0,"metric_values":"the archive's strings, untouched"},"metrics":["F1 score"],"metric_direction":{"note":"inferred from the metric name only (the archive records no direction); null = not inferred, chart draws points only","by_metric":{"F1 score":"higher"}},"counts":{"rows":27,"rows_with_code":25,"rows_with_paper_page":27,"rows_dated":27,"rows_using_additional_data":2},"rows":[{"rank_in_archive_order":1,"model":"Hashing + XLNet","metrics":{"F1 score":"96.43"},"uses_additional_data":false,"paper_date":"2024-06-12","paper":"/paper/to-be-continuous-or-to-be-discrete-those-are","paper_url":"https://arxiv.org/abs/2406.07812v1","paper_title":"To be Continuous, or to be Discrete, Those are Bits of Questions","code":"https://github.com/speedcell4/parserker","n_code_links":1,"syntology":null},{"rank_in_archive_order":2,"model":"SAPar + XLNet","metrics":{"F1 score":"96.40"},"uses_additional_data":false,"paper_date":"2020-10-15","paper":"/paper/improving-constituency-parsing-with-span","paper_url":"https://arxiv.org/abs/2010.07543v1","paper_title":"Improving Constituency Parsing with Span Attention","code":"https://github.com/cuhksz-nlp/SAPar","n_code_links":1,"syntology":{"n_ran":1,"n_unverified":0,"n_samples":1,"n_pointer_only_licence":0}},{"rank_in_archive_order":3,"model":"Label Attention Layer + HPSG + XLNet","metrics":{"F1 score":"96.38"},"uses_additional_data":false,"paper_date":"2019-11-10","paper":"/paper/rethinking-self-attention-an-interpretable","paper_url":"https://arxiv.org/abs/1911.03875v3","paper_title":"Rethinking Self-Attention: Towards Interpretability in Neural Parsing","code":"https://github.com/KhalilMrini/LAL-Parser","n_code_links":2,"syntology":{"n_ran":1,"n_unverified":2,"n_samples":3,"n_pointer_only_licence":3}},{"rank_in_archive_order":4,"model":"Attach-Juxtapose Parser + XLNet","metrics":{"F1 score":"96.34"},"uses_additional_data":false,"paper_date":"2020-10-27","paper":"/paper/strongly-incremental-constituency-parsing","paper_url":"https://arxiv.org/abs/2010.14568v1","paper_title":"Strongly Incremental Constituency Parsing with Graph Neural Networks","code":"https://github.com/yzhangcs/parser","n_code_links":3,"syntology":{"n_ran":2,"n_unverified":3,"n_samples":5,"n_pointer_only_licence":0}},{"rank_in_archive_order":5,"model":"Head-Driven Phrase Structure Grammar Parsing (Joint) + XLNet","metrics":{"F1 score":"96.33"},"uses_additional_data":false,"paper_date":"2019-07-05","paper":"/paper/head-driven-phrase-structure-grammar-parsing","paper_url":"https://arxiv.org/abs/1907.02684v4","paper_title":"Head-Driven Phrase Structure Grammar Parsing on Penn Treebank","code":"https://github.com/DoodleJZ/HPSG-Neural-Parser","n_code_links":1,"syntology":{"n_ran":2,"n_unverified":7,"n_samples":9,"n_pointer_only_licence":0}},{"rank_in_archive_order":6,"model":"CRF Parser + RoBERTa","metrics":{"F1 score":"96.32"},"uses_additional_data":false,"paper_date":"2020-08-09","paper":"/paper/fast-and-accurate-neural-crf-constituency-1","paper_url":"https://arxiv.org/abs/2008.03736v1","paper_title":"Fast and Accurate Neural CRF Constituency Parsing","code":"https://github.com/yzhangcs/parser","n_code_links":2,"syntology":{"n_ran":3,"n_unverified":0,"n_samples":3,"n_pointer_only_licence":0}},{"rank_in_archive_order":7,"model":"Hashing + Bert","metrics":{"F1 score":"96.03"},"uses_additional_data":false,"paper_date":"2024-06-12","paper":"/paper/to-be-continuous-or-to-be-discrete-those-are","paper_url":"https://arxiv.org/abs/2406.07812v1","paper_title":"To be Continuous, or to be Discrete, Those are Bits of Questions","code":"https://github.com/speedcell4/parserker","n_code_links":1,"syntology":null},{"rank_in_archive_order":8,"model":"N-ary semi-markov + BERT-large","metrics":{"F1 score":"95.92"},"uses_additional_data":false,"paper_date":"2021-07-26","paper":"/paper/n-ary-constituent-tree-parsing-with-recursive","paper_url":"https://aclanthology.org/2021.acl-long.205/","paper_title":"N-ary Constituent Tree Parsing with Recursive Semi-Markov Model","code":"https://github.com/NP-NET-research/Recursive-Semi-Markov-Model","n_code_links":1,"syntology":null},{"rank_in_archive_order":9,"model":"NFC + BERT-large","metrics":{"F1 score":"95.92"},"uses_additional_data":false,"paper_date":"2021-09-27","paper":"/paper/investigating-non-local-features-for-neural","paper_url":"https://arxiv.org/abs/2109.12814v2","paper_title":"Investigating Non-local Features for Neural Constituency Parsing","code":"https://github.com/ringos/nfc-parser","n_code_links":1,"syntology":null},{"rank_in_archive_order":10,"model":"Head-Driven Phrase Structure Grammar Parsing (Joint) + BERT","metrics":{"F1 score":"95.84"},"uses_additional_data":false,"paper_date":"2019-07-05","paper":"/paper/head-driven-phrase-structure-grammar-parsing","paper_url":"https://arxiv.org/abs/1907.02684v4","paper_title":"Head-Driven Phrase Structure Grammar Parsing on Penn Treebank","code":"https://github.com/DoodleJZ/HPSG-Neural-Parser","n_code_links":1,"syntology":{"n_ran":2,"n_unverified":7,"n_samples":9,"n_pointer_only_licence":0}},{"rank_in_archive_order":11,"model":"CRF Parser + BERT","metrics":{"F1 score":"95.69"},"uses_additional_data":false,"paper_date":"2020-08-09","paper":"/paper/fast-and-accurate-neural-crf-constituency-1","paper_url":"https://arxiv.org/abs/2008.03736v1","paper_title":"Fast and Accurate Neural CRF Constituency Parsing","code":"https://github.com/yzhangcs/parser","n_code_links":2,"syntology":{"n_ran":3,"n_unverified":0,"n_samples":3,"n_pointer_only_licence":0}},{"rank_in_archive_order":12,"model":"CNN Large + fine-tune","metrics":{"F1 score":"95.6"},"uses_additional_data":false,"paper_date":"2019-03-19","paper":"/paper/cloze-driven-pretraining-of-self-attention","paper_url":"http://arxiv.org/abs/1903.07785v1","paper_title":"Cloze-driven Pretraining of Self-attention Networks","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":13,"model":"SpanRel","metrics":{"F1 score":"95.5"},"uses_additional_data":false,"paper_date":"2019-11-10","paper":"/paper/generalizing-natural-language-analysis-1","paper_url":"https://arxiv.org/abs/1911.03822v2","paper_title":"Generalizing Natural Language Analysis through Span-relation Representations","code":"https://github.com/jzbjyb/SpanRel","n_code_links":3,"syntology":null},{"rank_in_archive_order":14,"model":"Tetra Tagging","metrics":{"F1 score":"95.44"},"uses_additional_data":false,"paper_date":"2019-04-22","paper":"/paper/tetra-tagging-word-synchronous-parsing-with","paper_url":"https://arxiv.org/abs/1904.09745v2","paper_title":"Tetra-Tagging: Word-Synchronous Parsing with Linear-Time Inference","code":"https://github.com/yzhangcs/parser","n_code_links":2,"syntology":null},{"rank_in_archive_order":15,"model":"Self-attentive encoder + ELMo","metrics":{"F1 score":"95.13"},"uses_additional_data":false,"paper_date":"2018-05-02","paper":"/paper/constituency-parsing-with-a-self-attentive","paper_url":"http://arxiv.org/abs/1805.01052v1","paper_title":"Constituency Parsing with a Self-Attentive Encoder","code":"https://github.com/nikitakit/self-attentive-parser","n_code_links":5,"syntology":{"n_ran":0,"n_unverified":14,"n_samples":14,"n_pointer_only_licence":0}},{"rank_in_archive_order":16,"model":"Model combination","metrics":{"F1 score":"94.66"},"uses_additional_data":false,"paper_date":"2017-07-10","paper":"/paper/improving-neural-parsing-by-disentangling","paper_url":"http://arxiv.org/abs/1707.03058v1","paper_title":"Improving Neural Parsing by Disentangling Model Combination and Reranking Effects","code":null,"n_code_links":0,"syntology":null},{"rank_in_archive_order":17,"model":"LSTM Encoder-Decoder + LSTM-LM","metrics":{"F1 score":"94.47"},"uses_additional_data":false,"paper_date":"2018-08-30","paper":"/paper/direct-output-connection-for-a-high-rank","paper_url":"http://arxiv.org/abs/1808.10143v2","paper_title":"Direct Output Connection for a High-Rank Language Model","code":"https://github.com/nttcslab-nlp/doc_lm","n_code_links":1,"syntology":null},{"rank_in_archive_order":18,"model":"LSTM Encoder-Decoder + LSTM-LM","metrics":{"F1 score":"94.32"},"uses_additional_data":false,"paper_date":"2018-07-01","paper":"/paper/an-empirical-study-of-building-a-strong","paper_url":"https://aclanthology.org/P18-2097","paper_title":"An Empirical Study of Building a Strong Baseline for Constituency Parsing","code":"https://github.com/nttcslab-nlp/strong_s2s_baseline_parser","n_code_links":1,"syntology":null},{"rank_in_archive_order":19,"model":"In-order","metrics":{"F1 score":"94.2"},"uses_additional_data":false,"paper_date":"2017-07-17","paper":"/paper/in-order-transition-based-constituent-parsing","paper_url":"http://arxiv.org/abs/1707.05000v1","paper_title":"In-Order Transition-based Constituent Parsing","code":"https://github.com/hantek/distance-parser","n_code_links":2,"syntology":null},{"rank_in_archive_order":20,"model":"CRF Parser","metrics":{"F1 score":"94.12"},"uses_additional_data":false,"paper_date":"2020-08-09","paper":"/paper/fast-and-accurate-neural-crf-constituency-1","paper_url":"https://arxiv.org/abs/2008.03736v1","paper_title":"Fast and Accurate Neural CRF Constituency Parsing","code":"https://github.com/yzhangcs/parser","n_code_links":2,"syntology":{"n_ran":3,"n_unverified":0,"n_samples":3,"n_pointer_only_licence":0}},{"rank_in_archive_order":21,"model":"Semi-supervised LSTM-LM","metrics":{"F1 score":"93.8"},"uses_additional_data":false,"paper_date":"2016-11-01","paper":"/paper/parsing-as-language-modeling","paper_url":"https://aclanthology.org/D16-1257","paper_title":"Parsing as Language Modeling","code":"https://github.com/cdg720/emnlp2016","n_code_links":1,"syntology":null},{"rank_in_archive_order":22,"model":"Stack-only RNNG","metrics":{"F1 score":"93.6"},"uses_additional_data":false,"paper_date":"2016-11-17","paper":"/paper/what-do-recurrent-neural-network-grammars","paper_url":"http://arxiv.org/abs/1611.05774v2","paper_title":"What Do Recurrent Neural Network Grammars Learn About Syntax?","code":"https://github.com/clab/rnng","n_code_links":1,"syntology":null},{"rank_in_archive_order":23,"model":"Transformer","metrics":{"F1 score":"92.7"},"uses_additional_data":false,"paper_date":"2017-06-12","paper":"/paper/attention-is-all-you-need","paper_url":"https://arxiv.org/abs/1706.03762v7","paper_title":"Attention Is All You Need","code":"https://github.com/huggingface/transformers","n_code_links":595,"syntology":{"n_ran":610,"n_unverified":336,"n_samples":946,"n_pointer_only_licence":451}},{"rank_in_archive_order":24,"model":"Parse fusion","metrics":{"F1 score":"92.6"},"uses_additional_data":true,"paper_date":"2015-09-01","paper":"/paper/syntactic-parse-fusion","paper_url":"https://aclanthology.org/D15-1160","paper_title":"Syntactic Parse Fusion","code":"https://github.com/BLLIP/bllip-parser","n_code_links":1,"syntology":null},{"rank_in_archive_order":25,"model":"Semi-supervised LSTM","metrics":{"F1 score":"92.1"},"uses_additional_data":false,"paper_date":"2014-12-23","paper":"/paper/grammar-as-a-foreign-language","paper_url":"http://arxiv.org/abs/1412.7449v3","paper_title":"Grammar as a Foreign Language","code":"https://github.com/atpaino/deep-text-corrector","n_code_links":7,"syntology":{"n_ran":0,"n_unverified":17,"n_samples":17,"n_pointer_only_licence":0}},{"rank_in_archive_order":26,"model":"Self-training","metrics":{"F1 score":"92.1"},"uses_additional_data":true,"paper_date":"2006-06-01","paper":"/paper/effective-self-training-for-parsing","paper_url":"https://www.researchgate.net/publication/262408350_Effective_self-training_for_parsing","paper_title":"Effective Self-Training for Parsing","code":"https://github.com/BLLIP/bllip-parser","n_code_links":1,"syntology":null},{"rank_in_archive_order":27,"model":"RNN Grammar","metrics":{"F1 score":"93.3"},"uses_additional_data":false,"paper_date":"2016-02-25","paper":"/paper/recurrent-neural-network-grammars","paper_url":"http://arxiv.org/abs/1602.07776v4","paper_title":"Recurrent Neural Network Grammars","code":"https://github.com/clab/rnng","n_code_links":6,"syntology":{"n_ran":6,"n_unverified":4,"n_samples":10,"n_pointer_only_licence":6}}],"since_archive":{"claim":"Results that newer papers report for their own method, placed here by Syntology. A model pointed at the cell in the paper's own table; the number was read from that cell and checked against this leaderboard's metric, dataset, split and scale; an independent check that saw this leaderboard's other rows and every other leaderboard on the same dataset accepted it. Not reviewed by the paper's authors or by the archive's editors, and not ranked against the archive rows.","extraction_file_present":true,"measurement":{"test_papers":883,"papers_with_output":881,"judged_true":108,"judged":110,"wilson95_lower":0.9361,"measured_on":"2026-09-24","frozen_commit":"0e3de0df94"},"measurement_note":"blind adjudication of accepted entries on a held-out split of archive papers, rules frozen before the test","coverage":{"sentence":"Syntology has checked 6,885 of the 9,623 papers on this site that are newer than the archive; results from the others appear after they are checked.","complete":false,"papers_newer_than_archive":9623,"papers_checked":6885,"papers_extracted_not_yet_verified":0,"boards_without_verdict":2,"papers_not_yet_extracted":2737},"order":"newest first by month (arXiv date, else the arXiv-id month), then arXiv id descending","columns":[],"entries":[]},"syntology":{"read_at":"2026-09-25T09:33:49+00:00","claim":"Per row: N of M harvested code samples from that row's paper executed on a synthesized fixture; the other M-N are unverified. Not a reproduction of the row's number; not a correctness claim. n_pointer_only_licence counts samples the site points at rather than redistributes (a licence axis, independent of ran/unverified).","rows_with_graph_line":12,"rows_with_any_sample_ran":10,"distinct_papers_with_graph_line":9,"distinct_papers_with_any_sample_ran":7,"samples_over_distinct_papers":{"n_ran":625,"n_unverified":383,"n_samples":1008,"n_pointer_only_licence":460,"note":"each paper (arXiv id) counted once, however many rows it is behind; this is the page-level figure"},"samples_row_weighted":{"n_ran":633,"n_unverified":390,"n_samples":1023,"n_pointer_only_licence":460,"note":"row-weighted: a paper behind several rows is counted once per row; inflated relative to samples_over_distinct_papers by design, kept for readers summing the per-row syntology blocks"}}}