{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/selective-vision-is-the-challenge-for-visual","title":"Selective Vision is the Challenge for Visual Reasoning: A Benchmark for Visual Argument Understanding","arxiv_id":"2406.18925","date":"2024-06-27","proceeding":null,"authors":["Jiwan Chung","Sungjae Lee","Minseo Kim","Seungju Han","Ashkan Yousefpour","Jack Hessel","Youngjae Yu"],"abstract":"Visual arguments, often used in advertising or social causes, rely on images to persuade viewers to do or believe something. Understanding these arguments requires selective vision: only specific visual stimuli within an image are relevant to the argument, and relevance can only be understood within the context of a broader argumentative structure. While visual arguments are readily appreciated by human audiences, we ask: are today's AI capable of similar understanding? We present VisArgs, a dataset of 1,611 images annotated with 5,112 visual premises (with regions), 5,574 commonsense premises, and reasoning trees connecting them into structured arguments. We propose three tasks for evaluating visual argument understanding: premise localization, premise identification, and conclusion deduction. Experiments show that 1) machines struggle to capture visual cues: GPT-4-O achieved 78.5% accuracy, while humans reached 98.0%. Models also performed 19.5% worse when distinguishing between irrelevant objects within the image compared to external objects. 2) Providing relevant visual premises improved model performance significantly.","url_abs":"https://arxiv.org/abs/2406.18925v3","url_pdf":"https://arxiv.org/pdf/2406.18925v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"selective-vision-is-the-challenge-for-visual","repo_url":"https://github.com/jiwanchung/visargs","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"visual-reasoning","task_name":"Visual Reasoning"}],"methods":[],"datasets_introduced":[{"slug":"visargs","name":"VisArgs","full_name":""}],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.18925","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.18925"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jiwanchung/visargs","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/JiwanChung/VisArgs","reach":{"status":"ok"}}],"summary":{"ran":6,"unverified":4},"by_repo_kind":{"official":{"samples":10,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":10,"samples":[{"code_sha256_prefix":"e82562fe176920d4","entry":"gen_option_prompt","repo":"JiwanChung/VisArgs","repo_kind":"official","path":"src/visarg/tasks/classification/classification.py","file_url":"https://github.com/JiwanChung/VisArgs/blob/HEAD/src/visarg/tasks/classification/classification.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e82562fe176920d4"}},{"code_sha256_prefix":"bf47774ade5b9148","entry":"identification_of_premises_prompt","repo":"JiwanChung/VisArgs","repo_kind":"official","path":"src/visarg/others/prompts.py","file_url":"https://github.com/JiwanChung/VisArgs/blob/HEAD/src/visarg/others/prompts.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bf47774ade5b9148"}},{"code_sha256_prefix":"6662fae2b3a506e9","entry":"interUnion","repo":"JiwanChung/VisArgs","repo_kind":"official","path":"src/visarg/others/utils.py","file_url":"https://github.com/JiwanChung/VisArgs/blob/HEAD/src/visarg/others/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6662fae2b3a506e9"}},{"code_sha256_prefix":"fe5e851bbfb6cb39","entry":"load_model","repo":"JiwanChung/VisArgs","repo_kind":"official","path":"src/visarg/tasks/classification/classification.py","file_url":"https://github.com/JiwanChung/VisArgs/blob/HEAD/src/visarg/tasks/classification/classification.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fe5e851bbfb6cb39"}},{"code_sha256_prefix":"825db40de691eea5","entry":"load_model","repo":"jiwanchung/visargs","repo_kind":"official","path":"src/visarg/tasks/localization/openset.py","file_url":"https://github.com/jiwanchung/visargs/blob/HEAD/src/visarg/tasks/localization/openset.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"825db40de691eea5"}},{"code_sha256_prefix":"9f4883895666b0aa","entry":"parse_result","repo":"JiwanChung/VisArgs","repo_kind":"official","path":"src/visarg/tasks/classification/classification.py","file_url":"https://github.com/JiwanChung/VisArgs/blob/HEAD/src/visarg/tasks/classification/classification.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9f4883895666b0aa"}},{"code_sha256_prefix":"69e0153f78e98e0d","entry":"eval_con","repo":"JiwanChung/VisArgs","repo_kind":"official","path":"src/visarg/others/eval_con.py","file_url":"https://github.com/JiwanChung/VisArgs/blob/HEAD/src/visarg/others/eval_con.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"69e0153f78e98e0d"}},{"code_sha256_prefix":"0f9943080d349652","entry":"load_model","repo":"JiwanChung/VisArgs","repo_kind":"official","path":"src/visarg/tasks/generation/generation.py","file_url":"https://github.com/JiwanChung/VisArgs/blob/HEAD/src/visarg/tasks/generation/generation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0f9943080d349652"}},{"code_sha256_prefix":"dbc27643d83f3e42","entry":"load_prompt_func","repo":"JiwanChung/VisArgs","repo_kind":"official","path":"src/visarg/tasks/generation/generation.py","file_url":"https://github.com/JiwanChung/VisArgs/blob/HEAD/src/visarg/tasks/generation/generation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dbc27643d83f3e42"}},{"code_sha256_prefix":"d3b76c2a8931dfce","entry":"save_image","repo":"JiwanChung/VisArgs","repo_kind":"official","path":"src/visarg/others/utils.py","file_url":"https://github.com/JiwanChung/VisArgs/blob/HEAD/src/visarg/others/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d3b76c2a8931dfce"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}