{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/the-amazing-mysteries-of-the-gutter-drawing","title":"The Amazing Mysteries of the Gutter: Drawing Inferences Between Panels in Comic Book Narratives","arxiv_id":"1611.05118","date":"2016-11-16","proceeding":"CVPR 2017 7","authors":["Mohit Iyyer","Varun Manjunatha","Anupam Guha","Yogarshi Vyas","Jordan Boyd-Graber","Hal Daumé III","Larry Davis"],"abstract":"Visual narrative is often a combination of explicit information and judicious\nomissions, relying on the viewer to supply missing details. In comics, most\nmovements in time and space are hidden in the \"gutters\" between panels. To\nfollow the story, readers logically connect panels together by inferring unseen\nactions through a process called \"closure\". While computers can now describe\nwhat is explicitly depicted in natural images, in this paper we examine whether\nthey can understand the closure-driven narratives conveyed by stylized artwork\nand dialogue in comic book panels. We construct a dataset, COMICS, that\nconsists of over 1.2 million panels (120 GB) paired with automatic textbox\ntranscriptions. An in-depth analysis of COMICS demonstrates that neither text\nnor image alone can tell a comic book story, so a computer must understand both\nmodalities to keep up with the plot. We introduce three cloze-style tasks that\nask models to predict narrative and character-centric aspects of a panel given\nn preceding panels as context. Various deep neural architectures underperform\nhuman baselines on these tasks, suggesting that COMICS contains fundamental\nchallenges for both vision and language.","url_abs":"http://arxiv.org/abs/1611.05118v2","url_pdf":"http://arxiv.org/pdf/1611.05118v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"the-amazing-mysteries-of-the-gutter-drawing","repo_url":"https://github.com/barisbatuhan/SSuperGAN","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"GPL-3.0"}},{"paper_slug":"the-amazing-mysteries-of-the-gutter-drawing","repo_url":"https://github.com/gsoykan/comics_text_plus","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"the-amazing-mysteries-of-the-gutter-drawing","repo_url":"https://github.com/miyyer/comics","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1611.05118","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1611.05118"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/miyyer/comics","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/barisbatuhan/SSuperGAN","reach":{"status":"ok","spdx":"GPL-3.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gsoykan/comics_text_plus","reach":{"status":"ok"}}],"summary":{"unverified":1},"by_repo_kind":{"listed":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"80f21d0245135d24","entry":"load_hdf5","repo":"miyyer/comics","repo_kind":"listed","path":"models/layers_and_utils.py","file_url":"https://github.com/miyyer/comics/blob/HEAD/models/layers_and_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"80f21d0245135d24"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}