{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-to-deceive-knowledge-graph-augmented-1","title":"Learning to Deceive Knowledge Graph Augmented Models via Targeted Perturbation","arxiv_id":"2010.12872","date":"2020-10-24","proceeding":"ICLR 2021 1","authors":["Mrigank Raman","Aaron Chan","Siddhant Agarwal","Peifeng Wang","Hansen Wang","Sungchul Kim","Ryan Rossi","Handong Zhao","Nedim Lipka","Xiang Ren"],"abstract":"Knowledge graphs (KGs) have helped neural models improve performance on various knowledge-intensive tasks, like question answering and item recommendation. By using attention over the KG, such KG-augmented models can also \"explain\" which KG information was most relevant for making a given prediction. In this paper, we question whether these models are really behaving as we expect. We show that, through a reinforcement learning policy (or even simple heuristics), one can produce deceptively perturbed KGs, which maintain the downstream performance of the original KG while significantly deviating from the original KG's semantics and structure. Our findings raise doubts about KG-augmented models' ability to reason about KG information and give sensible explanations.","url_abs":"https://arxiv.org/abs/2010.12872v6","url_pdf":"https://arxiv.org/pdf/2010.12872v6.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-to-deceive-knowledge-graph-augmented-1","repo_url":"https://github.com/INK-USC/deceive-KG-models","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"knowledge-graphs","task_name":"Knowledge Graphs"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"recommendation-systems","task_name":"Recommendation Systems"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2010.12872","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}