{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/certified-robustness-to-adversarial-word","title":"Certified Robustness to Adversarial Word Substitutions","arxiv_id":"1909.00986","date":"2019-09-03","proceeding":"IJCNLP 2019 11","authors":["Robin Jia","aditi raghunathan","Kerem Göksel","Percy Liang"],"abstract":"State-of-the-art NLP models can often be fooled by adversaries that apply seemingly innocuous label-preserving transformations (e.g., paraphrasing) to input text. The number of possible transformations scales exponentially with text length, so data augmentation cannot cover all transformations of an input. This paper considers one exponentially large family of label-preserving transformations, in which every word in the input can be replaced with a similar word. We train the first models that are provably robust to all word substitutions in this family. Our training procedure uses Interval Bound Propagation (IBP) to minimize an upper bound on the worst-case loss that any combination of word substitutions can induce. To evaluate models' robustness to these transformations, we measure accuracy on adversarially chosen word substitutions applied to test examples. Our IBP-trained models attain $75\\%$ adversarial accuracy on both sentiment analysis on IMDB and natural language inference on SNLI. In comparison, on IMDB, models trained normally and ones trained with data augmentation achieve adversarial accuracy of only $8\\%$ and $35\\%$, respectively.","url_abs":"https://arxiv.org/abs/1909.00986v1","url_pdf":"https://arxiv.org/pdf/1909.00986v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"certified-robustness-to-adversarial-word","repo_url":"https://worksheets.codalab.org/worksheets/0x79feda5f1998497db75422eca8fcd689","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null},{"paper_slug":"certified-robustness-to-adversarial-word","repo_url":"https://github.com/robinjia/certified-word-sub","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"natural-language-inference","task_name":"Natural Language Inference"},{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1909.00986","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1909.00986"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/robinjia/certified-word-sub","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://worksheets.codalab.org/worksheets/0x79feda5f1998497db75422eca8fcd689","reach":null}],"summary":{"unverified":3},"by_repo_kind":{"listed":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ab6c8d85177396c2","entry":"add","repo":"robinjia/certified-word-sub","repo_kind":"listed","path":"src/ibp.py","file_url":"https://github.com/robinjia/certified-word-sub/blob/HEAD/src/ibp.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ab6c8d85177396c2"}},{"code_sha256_prefix":"e17b7f13732b4b61","entry":"div","repo":"robinjia/certified-word-sub","repo_kind":"listed","path":"src/ibp.py","file_url":"https://github.com/robinjia/certified-word-sub/blob/HEAD/src/ibp.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e17b7f13732b4b61"}},{"code_sha256_prefix":"32e94bdfe3f0afad","entry":"mul","repo":"robinjia/certified-word-sub","repo_kind":"listed","path":"src/ibp.py","file_url":"https://github.com/robinjia/certified-word-sub/blob/HEAD/src/ibp.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"32e94bdfe3f0afad"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}