Papers › Storytelling from an Image Stream Using Scene Graphs
Storytelling from an Image Stream Using Scene Graphs
Ruize Wang, Zhongyu Wei, Piji Li, Qi Zhang, Xuanjing Huang
Visual storytelling aims at generating a story from an image stream. Most existing methods tend to represent images directly with the extracted high-level features, which is not intuitive and difficult to interpret. We argue that translating each image into a graph-based semantic representation, i.e., scene graph, which explicitly encodes the objects and relationships detected within image, would benefit representing and describing images. To this end, we propose a novel graph-based architecture for visual storytelling by modeling the two-level relationships on scene graphs. In particular, on the within-image level, we employ a Graph Convolution Network (GCN) to enrich local fine-grained region representations of objects on scene graphs. To further model the interaction among images, on the cross-images level, a Temporal Convolution Network (TCN) is utilized to refine the region representations along the temporal dimension. Then the relation-aware representations are fed into the Gated Recurrent Unit (GRU) with attention mechanism for story generation. Experiments are conducted on the public visual storytelling dataset. Automatic and human evaluation results indicate that our method achieves state-of-the-art.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Visual Storytelling | VIST | SGVST | BLEU-4 | 14.7 | #8 of 33 | Archive leaderboard | report |
| Visual Storytelling | VIST | SGVST | CIDEr | 9.8 | #8 of 33 | Archive leaderboard | report |
| Visual Storytelling | VIST | SGVST | METEOR | 35.8 | #8 of 33 | Archive leaderboard | report |
| Visual Storytelling | VIST | SGVST | ROUGE-L | 29.9 | #8 of 33 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections