{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/lm-critic-language-models-for-unsupervised","title":"LM-Critic: Language Models for Unsupervised Grammatical Error Correction","arxiv_id":"2109.06822","date":"2021-09-14","proceeding":"EMNLP 2021 11","authors":["Michihiro Yasunaga","Jure Leskovec","Percy Liang"],"abstract":"Training a model for grammatical error correction (GEC) requires a set of labeled ungrammatical / grammatical sentence pairs, but manually annotating such pairs can be expensive. Recently, the Break-It-Fix-It (BIFI) framework has demonstrated strong results on learning to repair a broken program without any labeled examples, but this relies on a perfect critic (e.g., a compiler) that returns whether an example is valid or not, which does not exist for the GEC task. In this work, we show how to leverage a pretrained language model (LM) in defining an LM-Critic, which judges a sentence to be grammatical if the LM assigns it a higher probability than its local perturbations. We apply this LM-Critic and BIFI along with a large set of unlabeled sentences to bootstrap realistic ungrammatical / grammatical pairs for training a corrector. We evaluate our approach on GEC datasets across multiple domains (CoNLL-2014, BEA-2019, GMEG-wiki and GMEG-yahoo) and show that it outperforms existing methods in both the unsupervised setting (+7.7 F0.5) and the supervised setting (+0.5 F0.5).","url_abs":"https://arxiv.org/abs/2109.06822v2","url_pdf":"https://arxiv.org/pdf/2109.06822v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"lm-critic-language-models-for-unsupervised","repo_url":"https://github.com/michiyasunaga/LM-Critic","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"lm-critic-language-models-for-unsupervised","repo_url":"https://github.com/elaynelemos/gector-api","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"lm-critic-language-models-for-unsupervised","repo_url":"https://github.com/grammarly/gector","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"grammatical-error-correction","task_name":"Grammatical Error Correction"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":null,"task_name":"valid"}],"methods":[{"method_slug":"repair","method_name":"Repair"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/grammatical-error-correction-on-bea-2019-test","task":"Grammatical Error Correction","dataset":"BEA-2019 (test)","model":"LM-Critic","rank_in_archive_order":12,"of":19,"metrics":{"F0.5":"72.9"},"uses_additional_data":false},{"leaderboard":"/sota/grammatical-error-correction-on-conll-2014","task":"Grammatical Error Correction","dataset":"CoNLL-2014 Shared Task","model":"LM-Critic","rank_in_archive_order":11,"of":23,"metrics":{"F0.5":"65.8"},"uses_additional_data":false},{"leaderboard":"/sota/grammatical-error-correction-on-restricted","task":"Grammatical Error Correction","dataset":"Restricted","model":"+ BIFI with no critic","rank_in_archive_order":4,"of":4,"metrics":{"F0.5":"18.7"},"uses_additional_data":false},{"leaderboard":"/sota/grammatical-error-correction-on-unrestricted","task":"Grammatical Error Correction","dataset":"Unrestricted","model":"+ BIFI (ours)","rank_in_archive_order":2,"of":3,"metrics":{"F0.5":"42.4"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2109.06822","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}