{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/viewpoint-invariant-change-captioning","title":"Robust Change Captioning","arxiv_id":"1901.02527","date":"2019-01-08","proceeding":"ICCV 2019 10","authors":["Dong Huk Park","Trevor Darrell","Anna Rohrbach"],"abstract":"Describing what has changed in a scene can be useful to a user, but only if\ngenerated text focuses on what is semantically relevant. It is thus important\nto distinguish distractors (e.g. a viewpoint change) from relevant changes\n(e.g. an object has moved). We present a novel Dual Dynamic Attention Model\n(DUDA) to perform robust Change Captioning. Our model learns to distinguish\ndistractors from semantic changes, localize the changes via Dual Attention over\n\"before\" and \"after\" images, and accurately describe them in natural language\nvia Dynamic Speaker, by adaptively focusing on the necessary visual inputs\n(e.g. \"before\" or \"after\" image). To study the problem in depth, we collect a\nCLEVR-Change dataset, built off the CLEVR engine, with 5 types of scene\nchanges. We benchmark a number of baselines on our dataset, and systematically\nstudy different change types and robustness to distractors. We show the\nsuperiority of our DUDA model in terms of both change captioning and\nlocalization. We also show that our approach is general, obtaining\nstate-of-the-art results on the recent realistic Spot-the-Diff dataset which\nhas no distractors.","url_abs":"http://arxiv.org/abs/1901.02527v2","url_pdf":"http://arxiv.org/pdf/1901.02527v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"viewpoint-invariant-change-captioning","repo_url":"https://github.com/Seth-Park/RobustChangeCaptioning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"natural-language-visual-grounding","task_name":"Natural Language Visual Grounding"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}