{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/adversarial-over-sensitivity-and-over","title":"Adversarial Over-Sensitivity and Over-Stability Strategies for Dialogue Models","arxiv_id":"1809.02079","date":"2018-09-06","proceeding":"CONLL 2018 10","authors":["Tong Niu","Mohit Bansal"],"abstract":"We present two categories of model-agnostic adversarial strategies that\nreveal the weaknesses of several generative, task-oriented dialogue models:\nShould-Not-Change strategies that evaluate over-sensitivity to small and\nsemantics-preserving edits, as well as Should-Change strategies that test if a\nmodel is over-stable against subtle yet semantics-changing modifications. We\nnext perform adversarial training with each strategy, employing a max-margin\napproach for negative generative examples. This not only makes the target\ndialogue model more robust to the adversarial inputs, but also helps it perform\nsignificantly better on the original inputs. Moreover, training on all\nstrategies combined achieves further improvements, achieving a new\nstate-of-the-art performance on the original task (also verified via human\nevaluation). In addition to adversarial training, we also address the\nrobustness task at the model-level, by feeding it subword units as both inputs\nand outputs, and show that the resulting model is equally competitive, requires\nonly 1/4 of the original vocabulary size, and is robust to one of the\nadversarial strategies (to which the original model is vulnerable) even without\nadversarial training.","url_abs":"http://arxiv.org/abs/1809.02079v1","url_pdf":"http://arxiv.org/pdf/1809.02079v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"adversarial-over-sensitivity-and-over","repo_url":"https://github.com/WolfNiu/AdversarialDialogue","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"sensitivity","task_name":"Sensitivity"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1809.02079","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1809.02079"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/WolfNiu/AdversarialDialogue","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":7},"by_repo_kind":{"official":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e878ba003537ff17","entry":"build_dict","repo":"WolfNiu/AdversarialDialogue","repo_kind":"official","path":"src/basic/util.py","file_url":"https://github.com/WolfNiu/AdversarialDialogue/blob/HEAD/src/basic/util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e878ba003537ff17"}},{"code_sha256_prefix":"1463fb622dd5abde","entry":"build_index2token","repo":"WolfNiu/AdversarialDialogue","repo_kind":"official","path":"src/basic/util.py","file_url":"https://github.com/WolfNiu/AdversarialDialogue/blob/HEAD/src/basic/util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1463fb622dd5abde"}},{"code_sha256_prefix":"950f8b6375fa3b09","entry":"convert_to_number","repo":"WolfNiu/AdversarialDialogue","repo_kind":"official","path":"src/model/util.py","file_url":"https://github.com/WolfNiu/AdversarialDialogue/blob/HEAD/src/model/util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"950f8b6375fa3b09"}},{"code_sha256_prefix":"1d29567f1b72bfd6","entry":"double","repo":"WolfNiu/AdversarialDialogue","repo_kind":"official","path":"src/model/util.py","file_url":"https://github.com/WolfNiu/AdversarialDialogue/blob/HEAD/src/model/util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1d29567f1b72bfd6"}},{"code_sha256_prefix":"f678595b260df788","entry":"sequence_loss","repo":"WolfNiu/AdversarialDialogue","repo_kind":"official","path":"src/model/util.py","file_url":"https://github.com/WolfNiu/AdversarialDialogue/blob/HEAD/src/model/util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f678595b260df788"}},{"code_sha256_prefix":"bfb118ac1dd2775b","entry":"split_lst","repo":"WolfNiu/AdversarialDialogue","repo_kind":"official","path":"src/basic/util.py","file_url":"https://github.com/WolfNiu/AdversarialDialogue/blob/HEAD/src/basic/util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bfb118ac1dd2775b"}},{"code_sha256_prefix":"ca020336382cc115","entry":"to_str","repo":"WolfNiu/AdversarialDialogue","repo_kind":"official","path":"src/basic/ubuntu_data_generator.py","file_url":"https://github.com/WolfNiu/AdversarialDialogue/blob/HEAD/src/basic/ubuntu_data_generator.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ca020336382cc115"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}