Browse State-of-the-Art › Visual Storytelling
Visual Storytelling
37 papers with code · 1 benchmark · 4 datasets archive 2025-07-28
( Image credit: No Metrics Are Perfect )
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
1 leaderboard table shown for this task, 1 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
| Dataset | Best model (first row in archive order) | Paper | Code | Syntology | Compare |
|---|---|---|---|---|---|
| VIST (33 rows) | HEGR | Two Heads are Better Than One: Hypergraph-Enhanced Graph Reasoning... | — | — | Compare |
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
4 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
1 subtask in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 115 · with a code link: 37 · where Syntology ran a sample: 4 (4 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (4 of 115 tagged: 4 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument)
Page 1 of 2: papers 1 to 100 of 115, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
27 Aug 2024 2 repositories listedDesigns and artworks are ubiquitous across various creative fields, requiring graphic design skills and dedicated software to create compositions that include many graphical elements, such as logos, icons, symbols, and…
-
1 Jan 2021 2 repositories listedVisual storytelling and story comprehension are uniquely human skills that play a central role in how we learn about and experience the world.
-
3 Jun 2018 2 repositories listedWe present a neural model for generating short stories from image sequences, which extends the image description model by Vinyals et al. (Vinyals et al., 2015).
-
24 Apr 2018 2 repositories listed Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 1 where Syntology's instrument failed) · 4 unverified (of 8 harvested samples) · 1 pointer-only (licence)Though impressive results have been achieved in visual captioning, the task of generating abstract stories from photo streams is still a little-tapped problem.
-
11 Jun 2025 1 repository listedText-to-image generation models have made significant progress in producing high-quality images from textual descriptions, yet they continue to struggle with maintaining subject consistency across multiple images, a…
-
StoryReasoning Dataset: Using Chain-of-Thought for Scene Understanding and Grounded Story Generation15 May 2025 1 repository listedVisual storytelling systems struggle to maintain character identity across frames and link actions to appropriate subjects, frequently leading to referential hallucinations.
-
16 Apr 2025 1 repository listedOver the past years, advances in artificial intelligence (AI) have demonstrated how AI can solve many perception and generation tasks, such as image classification and text writing, yet reasoning remains a challenge.
-
16 Nov 2024 1 repository listedSketch animations offer a powerful medium for visual storytelling, from simple flip-book doodles to professional studio productions.
-
8 Oct 2024 1 repository listedThis tutorial explores recent advancements in multimodal pretrained and large models, capable of integrating and processing diverse data forms such as text, images, audio, and video.
-
7 Aug 2024 1 repository listedRecent image generation models excel at creating high-quality images from brief captions.
-
13 Jul 2024 1 repository listedVisual storytelling involves generating a sequence of coherent frames from a textual storyline while maintaining consistency in characters and scenes.
-
5 Jul 2024 1 repository listedWe then use this method to evaluate the stories generated by several models, showing that the foundation model LLaVA obtains the best result, but only slightly so compared to TAPM, a 50-times smaller visual storytelling…
-
15 Jun 2024 1 repository listedTo address this gap, we introduce CoMM, a high-quality Coherent interleaved image-text MultiModal dataset designed to enhance the coherence, consistency, and alignment of generated multimodal content.
-
22 Apr 2024 1 repository listedContemporary makeup transfer methods primarily focus on replicating makeup from one face to another, considerably limiting their use in creating diverse and creative character makeup essential for visual storytelling.
-
1 Jan 2024 1 repository listedGenerative models have recently exhibited exceptional capabilities in text-to-image generation but still struggle to generate image sequences coherently.
-
3 Nov 2023 1 repository listedYet, the desire to experience manga in vibrant colors has sparked the pursuit of manga colorization, a task of paramount significance for artists.
-
26 Oct 2023 1 repository listed Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples) · 3 pointer-only (licence)A proper evaluation of stories generated for a sequence of images -- the task commonly referred to as visual storytelling -- must consider multiple aspects, such as coherence, grammatical correctness, and visual…
-
6 Oct 2023 1 repository listedIn this paper, we collect an anthology of 100 visual stories from authors who participated in our systematic creative process of improvised story-building based on image sequences.
-
31 Aug 2023 1 repository listedLarge vision-language models (LVLMs) have recently witnessed rapid advancements, exhibiting a remarkable capacity for perceiving, understanding, and processing visual information by connecting visual receptor with large…
-
13 Jul 2023 1 repository listedFor the first module, we leverage an off-the-shelf video retrieval system and extract video depths as motion structure.
-
1 Jun 2023 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 3 harvested samples)Generative models have recently exhibited exceptional capabilities in text-to-image generation, but still struggle to generate image sequences coherently.
-
3 May 2023 1 repository listedHumans can naturally reason from superficial state differences (e. g. ground wetness) to transformations descriptions (e. g. raining) according to their life experience.
-
30 Mar 2023 1 repository listedCharacters are essential to the plot of any story.
-
20 Mar 2023 1 repository listedWe present Positional Diffusion, a plug-and-play graph formulation with Diffusion Probabilistic Models to address positional reasoning.
-
1 Jul 2022 1 repository listedWe measure the reliability of our metric sets by analysing its correlation with human judgement scores on a sample of machine stories obtained from 4 state-of-the-arts models trained on the Visual Storytelling Dataset…
-
31 May 2022 1 repository listedThese results depict the effectiveness of commonsense knowledge infusion in improving the performance and expressiveness of scene graph generation for visual understanding and reasoning tasks.
-
8 May 2022 1 repository listedWe measure the reliability of our metric sets by analysing its correlation with human judgement scores on a sample of machine stories obtained from 4 state-of-the-arts models trained on the Visual Storytelling Dataset…
-
1 May 2022 1 repository listedIn this paper, we present the VHED (VIST Human Evaluation Data) dataset, which first re-purposes human evaluation results for automatic evaluation; hence we develop Vrank (VIST Ranker), a novel reference-free VIST…
-
14 May 2021 1 repository listedWriting a coherent and engaging story is not easy.
-
3 Dec 2019 1 repository listedThis paper introduces KG-Story, a three-stage framework that allows the story generation model to take advantage of external Knowledge Graphs to produce interesting stories.
-
11 Sep 2019 1 repository listed Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 10 with no instrument failure: 0 honoured, 0 violated, 10 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 12 harvested samples)
-
7 Jul 2019 1 repository listed
-
5 Jun 2019 1 repository listed
-
4 May 2019 1 repository listed
-
6 Mar 2019 1 repository listed
-
28 May 2018 1 repository listed
-
13 Apr 2016 1 repository listed
-
Shape2Animal: Creative Animal Generation from Natural Silhouettes25 Jun 2025 0 repositories listed
-
JarvisArt: Liberating Human Artistic Creativity via an Intelligent Photo Retouching Agent21 Jun 2025 0 repositories listed
-
Camera Trajectory Generation: A Comprehensive Survey of Methods, Metrics, and Future Directions1 Jun 2025 0 repositories listed
-
LoRAShop: Training-Free Multi-Concept Image Generation and Editing with Rectified Flow Transformers29 May 2025 0 repositories listed
-
Action2Dialogue: Generating Character-Centric Narratives from Scene-Level Prompts22 May 2025 0 repositories listed
-
VIST-GPT: Ushering in the Era of Visual Storytelling with LLMs?27 Apr 2025 0 repositories listed
-
GenDoP: Auto-regressive Camera Trajectory Generation as a Director of Photography9 Apr 2025 0 repositories listed
-
Storybooth: Training-free Multi-Subject Consistency for Improved Visual Storytelling8 Apr 2025 0 repositories listed
-
DANTE-AD: Dual-Vision Attention Network for Long-Term Audio Description31 Mar 2025 0 repositories listed
-
VinaBench: Benchmark for Faithful and Consistent Visual Narratives26 Mar 2025 0 repositories listed
-
MIRAGE: Multimodal Immersive Reasoning and Guided Exploration for Red-Team Jailbreak Attacks24 Mar 2025 0 repositories listed
-
Toyteller: AI-powered Visual Storytelling Through Toy-Playing with Character Symbols23 Jan 2025 0 repositories listed
-
Generative Visual Communication in the Era of Vision-Language Models27 Nov 2024 0 repositories listed
-
A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Tasks9 Nov 2024 0 repositories listed
-
KAHANI: Culturally-Nuanced Visual Storytelling Pipeline for Non-Western Cultures25 Oct 2024 0 repositories listed
-
Generating Visual Stories with Grounded and Coreferent Characters20 Sep 2024 0 repositories listed
-
Semantic Alignment for Multimodal Large Language Models23 Aug 2024 0 repositories listed
-
Story3D-Agent: Exploring 3D Storytelling Visualization with Large Language Models21 Aug 2024 0 repositories listed
-
Context-aware Visual Storytelling with Visual Prefix Tuning and Contrastive Learning12 Aug 2024 0 repositories listed
-
Improving Visual Storytelling with Multimodal Large Language Models2 Jul 2024 0 repositories listed
-
TARN-VIST: Topic Aware Reinforcement Network for Visual Storytelling18 Mar 2024 0 repositories listed
-
AesopAgent: Agent-driven Evolutionary System on Story-to-Video Production12 Mar 2024 0 repositories listed
-
Metamorpheus: Interactive, Affective, and Creative Dream Narration Through Metaphorical Visual Storytelling1 Mar 2024 0 repositories listed
-
SCO-VIST: Social Interaction Commonsense Knowledge-based Visual Storytelling1 Feb 2024 0 repositories listed
-
MagicScroll: Nontypical Aspect-Ratio Image Generation for Visual Storytelling via Multi-Layered Semantic-Aware Denoising18 Dec 2023 0 repositories listed
-
DiffuVST: Narrating Fictional Scenes with Global-History-Guided Denoising Models12 Dec 2023 0 repositories listed
-
Visual Storytelling with Question-Answer Plans8 Oct 2023 0 repositories listed
-
Comics for Everyone: Generating Accessible Text Descriptions for Comic Strips1 Oct 2023 0 repositories listed
-
Text-Only Training for Visual Storytelling17 Aug 2023 0 repositories listed
-
26 Jun 2023 0 repositories listed
-
Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillings3 May 2023 0 repositories listed
-
A-CAP: Anticipation Captioning with Commonsense Knowledge13 Apr 2023 0 repositories listed
-
20 Jan 2023 0 repositories listed
-
A survey on knowledge-enhanced multimodal learning19 Nov 2022 0 repositories listed
-
DiMBERT: Learning Vision-Language Grounded Representations with Disentangled Multimodal-Attention28 Oct 2022 0 repositories listed
-
Bloom Library: Multimodal Datasets in 300+ Languages for a Variety of Downstream Tasks26 Oct 2022 0 repositories listed
-
6 Oct 2022 0 repositories listed
-
17 Aug 2022 0 repositories listed
-
16 Jun 2022 0 repositories listed
-
Knowledge-enriched Attention Network with Group-wise Semantic for Visual Storytelling10 Mar 2022 0 repositories listed
-
A System for Image Understanding using Sensemaking and Narrative21 Jan 2022 0 repositories listed
-
Discourse Analysis for Evaluating Coherence in Video Paragraph Captions17 Jan 2022 0 repositories listed
-
10 Jan 2022 0 repositories listed
-
RoViST: Learning Robust Metrics for Visual Storytelling17 Dec 2021 0 repositories listed
-
Learning to Rank Visual Stories From Human Ranking Data16 Nov 2021 0 repositories listed
-
Towards Coherent Visual Storytelling with Ordered Image Attention16 Nov 2021 0 repositories listed
-
Graph Similarities and Dual Approach for Sequential Text-to-Image Retrieval29 Sep 2021 0 repositories listed
-
Ordered Attention for Coherent Visual Storytelling4 Aug 2021 0 repositories listed
-
Stretch-VST: Getting Flexible With Visual Stories1 Aug 2021 0 repositories listed
-
18 Jul 2021 0 repositories listed
-
19 Jun 2021 0 repositories listed
-
18 May 2021 0 repositories listed
-
5 Feb 2021 0 repositories listed
-
3 Dec 2020 0 repositories listed
-
1 Nov 2020 0 repositories listed
-
Reading Between the Lines: Exploring Infilling in Visual Narratives26 Oct 2020 0 repositories listed
-
1 Sep 2020 0 repositories listed
-
Topic Adaptation and Prototype Encoding for Few-Shot Visual Storytelling11 Aug 2020 0 repositories listed
-
3 Apr 2020 0 repositories listed
-
3 Feb 2020 0 repositories listed
-
13 Jan 2020 0 repositories listed
-
Incorporating Textual Evidence in Visual Storytelling21 Nov 2019 0 repositories listed
-
11 Nov 2019 0 repositories listed
Syntology lines on 4 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections