{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/automatic-annotation-and-evaluation-of-error","title":"Automatic Annotation and Evaluation of Error Types for Grammatical Error Correction","arxiv_id":null,"date":"2017-07-01","proceeding":"ACL 2017 7","authors":["Christopher Bryant","Mariano Felice","Ted Briscoe"],"abstract":"Until now, error type performance for Grammatical Error Correction (GEC) systems could only be measured in terms of recall because system output is not annotated. To overcome this problem, we introduce ERRANT, a grammatical ERRor ANnotation Toolkit designed to automatically extract edits from parallel original and corrected sentences and classify them according to a new, dataset-agnostic, rule-based framework. This not only facilitates error type evaluation at different levels of granularity, but can also be used to reduce annotator workload and standardise existing GEC datasets. Human experts rated the automatic edits as {``}Good{''} or {``}Acceptable{''} in at least 95{\\%} of cases, so we applied ERRANT to the system output of the CoNLL-2014 shared task to carry out a detailed error type analysis for the first time.","url_abs":"https://aclanthology.org/P17-1074","url_pdf":"https://aclanthology.org/P17-1074.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"automatic-annotation-and-evaluation-of-error","repo_url":"https://github.com/chrisjbryant/errant","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"annotated-code-search","task_name":"Annotated Code Search"},{"task_slug":"grammatical-error-correction","task_name":"Grammatical Error Correction"},{"task_slug":"news-annotation","task_name":"News Annotation"},{"task_slug":"table-annotation","task_name":"Table annotation"},{"task_slug":"type","task_name":"Vocal Bursts Type Prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}