Papers › Hierarchical Photo-Scene Encoder for Album Storytelling
Hierarchical Photo-Scene Encoder for Album Storytelling
Bairui Wang, Lin Ma, Wei zhang, Wenhao Jiang, Feng Zhang
In this paper, we propose a novel model with a hierarchical photo-scene encoder and a reconstructor for the task of album storytelling. The photo-scene encoder contains two sub-encoders, namely the photo and scene encoders, which are stacked together and behave hierarchically to fully exploit the structure information of the photos within an album. Specifically, the photo encoder generates semantic representation for each photo while exploiting temporal relationships among them. The scene encoder, relying on the obtained photo representations, is responsible for detecting the scene changes and generating scene representations. Subsequently, the decoder dynamically and attentively summarizes the encoded photo and scene representations to generate a sequence of album representations, based on which a story consisting of multiple coherent sentences is generated. In order to fully extract the useful semantic information from an album, a reconstructor is employed to reproduce the summarized album representations based on the hidden states of the decoder. The proposed model can be trained in an end-to-end manner, which results in an improved performance over the state-of-the-arts on the public visual storytelling (VIST) dataset. Ablation studies further demonstrate the effectiveness of the proposed hierarchical photo-scene encoder and reconstructor.
In Syntology View this paper on Syntology: its page in Syntology's graph, with its repositories and citations.
Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Image-guided Story Ending Generation | VIST-E | T-CVAE | BLEU-1 | 14.34 | #5 of 6 | Archive leaderboard | report |
| Image-guided Story Ending Generation | VIST-E | T-CVAE | BLEU-2 | 5.06 | #5 of 6 | Archive leaderboard | report |
| Image-guided Story Ending Generation | VIST-E | T-CVAE | BLEU-3 | 2.01 | #5 of 6 | Archive leaderboard | report |
| Image-guided Story Ending Generation | VIST-E | T-CVAE | BLEU-4 | 1.13 | #5 of 6 | Archive leaderboard | report |
| Image-guided Story Ending Generation | VIST-E | T-CVAE | CIDEr | 11.49 | #5 of 6 | Archive leaderboard | report |
| Image-guided Story Ending Generation | VIST-E | T-CVAE | METEOR | 4.23 | #5 of 6 | Archive leaderboard | report |
| Image-guided Story Ending Generation | VIST-E | T-CVAE | ROUGE-L | 15.51 | #5 of 6 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections