Papers › Paying More Attention to Saliency: Image Captioning with Saliency and Context Attention
Paying More Attention to Saliency: Image Captioning with Saliency and Context Attention
Marcella Cornia, Lorenzo Baraldi, Giuseppe Serra, Rita Cucchiara
Image captioning has been recently gaining a lot of attention thanks to the impressive achievements shown by deep captioning architectures, which combine Convolutional Neural Networks to extract image representations, and Recurrent Neural Networks to generate the corresponding captions. At the same time, a significant research effort has been dedicated to the development of saliency prediction models, which can predict human eye fixations. Even though saliency information could be useful to condition an image captioning architecture, by providing an indication of what is salient and what is not, research is still struggling to incorporate these two techniques. In this work, we propose an image captioning approach in which a generative recurrent neural network can focus on different parts of the input image during the generation of the caption, by exploiting the conditioning given by a saliency prediction model on which parts of the image are salient and which are contextual. We show, through extensive quantitative and qualitative experiments on large scale datasets, that our model achieves superior performances with respect to captioning baselines with and without saliency, and to different state of the art approaches combining saliency and captioning.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Image Captioning | Flickr30k Captions test | Cornia et al | BLEU-4 | 21.3 | #2 of 7 | Archive leaderboard | report |
| Image Captioning | Flickr30k Captions test | Cornia et al | CIDEr | 46.4 | #2 of 7 | Archive leaderboard | report |
| Image Captioning | Flickr30k Captions test | Cornia et al | METEOR | 20.0 | #2 of 7 | Archive leaderboard | report |
| Image Captioning | Flickr30k Captions test | Cornia et al | SPICE | - | #2 of 7 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections