Visual Storytelling

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

4 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

1 subtask in the archive's task tree.

Parent tasks archive 2025-07-28

Most implemented papers archive 2025-07-28

30 shown of 37 papers with code (115 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Alfie: Democratising RGBA Image Generation With No $$$
  27 Aug 2024 2 repositories listed
  Designs and artworks are ubiquitous across various creative fields, requiring graphic design skills and dedicated software to create compositions that include many graphical elements, such as logos, icons, symbols, and…
• AESOP: Abstract Encoding of Stories, Objects, and Pictures
  1 Jan 2021 2 repositories listed
  Visual storytelling and story comprehension are uniquely human skills that play a central role in how we learn about and experience the world.
• Contextualize, Show and Tell: A Neural Visual Storyteller
  3 Jun 2018 2 repositories listed
  We present a neural model for generating short stories from image sequences, which extends the image description model by Vinyals et al.
• No Metrics Are Perfect: Adversarial Reward Learning for Visual Storytelling
  24 Apr 2018 2 repositories listed Syntology ran 1 of 8 samples · 7 unverified
  Though impressive results have been achieved in visual captioning, the task of generating abstract stories from photo streams is still a little-tapped problem.
• Consistent Story Generation with Asymmetry Zigzag Sampling
  11 Jun 2025 1 repository listed
  Text-to-image generation models have made significant progress in producing high-quality images from textual descriptions, yet they continue to struggle with maintaining subject consistency across multiple images, a…
• StoryReasoning Dataset: Using Chain-of-Thought for Scene Understanding and Grounded Story Generation
  15 May 2025 1 repository listed
  Visual storytelling systems struggle to maintain character identity across frames and link actions to appropriate subjects, frequently leading to referential hallucinations.
• FLIP Reasoning Challenge
  16 Apr 2025 1 repository listed
  Over the past years, advances in artificial intelligence (AI) have demonstrated how AI can solve many perception and generation tasks, such as image classification and text writing, yet reasoning remains a challenge.
• FlipSketch: Flipping Static Drawings to Text-Guided Sketch Animations
  16 Nov 2024 1 repository listed
  Sketch animations offer a powerful medium for visual storytelling, from simple flip-book doodles to professional studio productions.
• Multimodal Large Language Models and Tunings: Vision, Language, Sensors, Audio, and Beyond
  8 Oct 2024 1 repository listed
  This tutorial explores recent advancements in multimodal pretrained and large models, capable of integrating and processing diverse data forms such as text, images, audio, and video.
• Openstory++: A Large-scale Dataset and Benchmark for Instance-aware Open-domain Visual Storytelling
  7 Aug 2024 1 repository listed
  Recent image generation models excel at creating high-quality images from brief captions.
• ContextualStory: Consistent Visual Storytelling with Spatially-Enhanced and Storyline Context
  13 Jul 2024 1 repository listed
  Visual storytelling involves generating a sequence of coherent frames from a textual storyline while maintaining consistency in characters and scenes.
• Not (yet) the whole story: Evaluating Visual Storytelling Requires More than Measuring Coherence, Grounding, and Repetition
  5 Jul 2024 1 repository listed
  We then use this method to evaluate the stories generated by several models, showing that the foundation model LLaVA obtains the best result, but only slightly so compared to TAPM, a 50-times smaller visual storytelling…
• CoMM: A Coherent Interleaved Image-Text Dataset for Multimodal Understanding and Generation
  15 Jun 2024 1 repository listed
  To address this gap, we introduce CoMM, a high-quality Coherent interleaved image-text MultiModal dataset designed to enhance the coherence, consistency, and alignment of generated multimodal content.
• Gorgeous: Create Your Desired Character Facial Makeup from Any Ideas
  22 Apr 2024 1 repository listed
  Contemporary makeup transfer methods primarily focus on replicating makeup from one face to another, considerably limiting their use in creating diverse and creative character makeup essential for visual storytelling.
• Intelligent Grimm - Open-ended Visual Storytelling via Latent Diffusion Models
  1 Jan 2024 1 repository listed
  Generative models have recently exhibited exceptional capabilities in text-to-image generation but still struggle to generate image sequences coherently.
• inkn'hue: Enhancing Manga Colorization from Multiple Priors with Alignment Multi-Encoder VAE
  3 Nov 2023 1 repository listed
  Yet, the desire to experience manga in vibrant colors has sparked the pursuit of manga colorization, a task of paramount significance for artists.
• GROOViST: A Metric for Grounding Objects in Visual Storytelling
  26 Oct 2023 1 repository listed Syntology ran 2 of 3 samples · 1 unverified · 3 pointer-only (licence)
  A proper evaluation of stories generated for a sequence of images -- the task commonly referred to as visual storytelling -- must consider multiple aspects, such as coherence, grammatical correctness, and visual…
• Envisioning Narrative Intelligence: A Creative Visual Storytelling Anthology
  6 Oct 2023 1 repository listed
  In this paper, we collect an anthology of 100 visual stories from authors who participated in our systematic creative process of improvised story-building based on image sequences.
• TouchStone: Evaluating Vision-Language Models by Language Models
  31 Aug 2023 1 repository listed
  Large vision-language models (LVLMs) have recently witnessed rapid advancements, exhibiting a remarkable capacity for perceiving, understanding, and processing visual information by connecting visual receptor with large…
• Animate-A-Story: Storytelling with Retrieval-Augmented Video Generation
  13 Jul 2023 1 repository listed
  For the first module, we leverage an off-the-shelf video retrieval system and extract video depths as motion structure.
• Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion Models
  1 Jun 2023 1 repository listed Syntology ran 0 of 3 samples · 3 unverified
  Generative models have recently exhibited exceptional capabilities in text-to-image generation, but still struggle to generate image sequences coherently.
• Visual Transformation Telling
  3 May 2023 1 repository listed
  Humans can naturally reason from superficial state differences (e.
• Detecting and Grounding Important Characters in Visual Stories
  30 Mar 2023 1 repository listed
  Characters are essential to the plot of any story.
• Positional Diffusion: Ordering Unordered Sets with Diffusion Probabilistic Models
  20 Mar 2023 1 repository listed
  We present Positional Diffusion, a plug-and-play graph formulation with Diffusion Probabilistic Models to address positional reasoning.
• RoViST: Learning Robust Metrics for Visual Storytelling
  1 Jul 2022 1 repository listed
  We measure the reliability of our metric sets by analysing its correlation with human judgement scores on a sample of machine stories obtained from 4 state-of-the-arts models trained on the Visual Storytelling Dataset…
• Expressive Scene Graph Generation Using Commonsense Knowledge Infusion for Visual Understanding and Reasoning
  31 May 2022 1 repository listed
  These results depict the effectiveness of commonsense knowledge infusion in improving the performance and expressiveness of scene graph generation for visual understanding and reasoning tasks.
• RoViST:Learning Robust Metrics for Visual Storytelling
  8 May 2022 1 repository listed
  We measure the reliability of our metric sets by analysing its correlation with human judgement scores on a sample of machine stories obtained from 4 state-of-the-arts models trained on the Visual Storytelling Dataset…
• Learning to Rank Visual Stories From Human Ranking Data
  1 May 2022 1 repository listed
  In this paper, we present the VHED (VIST Human Evaluation Data) dataset, which first re-purposes human evaluation results for automatic evaluation; hence we develop Vrank (VIST Ranker), a novel reference-free VIST…
• Plot and Rework: Modeling Storylines for Visual Storytelling
  14 May 2021 1 repository listed
  Writing a coherent and engaging story is not easy.
• Knowledge-Enriched Visual Storytelling
  3 Dec 2019 1 repository listed
  This paper introduces KG-Story, a three-stage framework that allows the story generation model to take advantage of external Knowledge Graphs to produce interesting stories.

Syntology lines on 3 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.