{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/paws-paraphrase-adversaries-from-word","title":"PAWS: Paraphrase Adversaries from Word Scrambling","arxiv_id":"1904.01130","date":"2019-04-01","proceeding":"NAACL 2019 6","authors":["Yuan Zhang","Jason Baldridge","Luheng He"],"abstract":"Existing paraphrase identification datasets lack sentence pairs that have\nhigh lexical overlap without being paraphrases. Models trained on such data\nfail to distinguish pairs like flights from New York to Florida and flights\nfrom Florida to New York. This paper introduces PAWS (Paraphrase Adversaries\nfrom Word Scrambling), a new dataset with 108,463 well-formed paraphrase and\nnon-paraphrase pairs with high lexical overlap. Challenging pairs are generated\nby controlled word swapping and back translation, followed by fluency and\nparaphrase judgments by human raters. State-of-the-art models trained on\nexisting datasets have dismal performance on PAWS (<40% accuracy); however,\nincluding PAWS training data for these models improves their accuracy to 85%\nwhile maintaining performance on existing tasks. In contrast, models that do\nnot capture non-local contextual information fail even with PAWS training\nexamples. As such, PAWS provides an effective instrument for driving further\nprogress on models that better exploit structure, context, and pairwise\ncomparisons.","url_abs":"http://arxiv.org/abs/1904.01130v1","url_pdf":"http://arxiv.org/pdf/1904.01130v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"paws-paraphrase-adversaries-from-word","repo_url":"https://github.com/SaubanMusaddiq/Paraphrase_Detection_on_PAWS","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"paws-paraphrase-adversaries-from-word","repo_url":"https://github.com/google-research-datasets/paws","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"paraphrase-identification","task_name":"Paraphrase Identification"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[{"slug":"paws","name":"PAWS","full_name":"Paraphrase Adversaries from Word Scrambling"}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1904.01130","atlas_url":"https://app.syntology.ai/?focus=1904.01130","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}