Papers › Show, Attend and Tell: Neural Image Caption Generation with Visual Attention

Show, Attend and Tell: Neural Image Caption Generation with Visual Attention

10 Feb 2015arXiv:1502.03044archive 2025-07-28

Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard Zemel, Yoshua Bengio

Inspired by recent work in machine translation and object detection, we introduce an attention based model that automatically learns to describe the content of images. We describe how we can train this model in a deterministic manner using standard backpropagation techniques and stochastically by maximizing a variational lower bound. We also show through visualization how the model is able to automatically learn to fix its gaze on salient objects while generating the corresponding words in the output sequence. We validate the use of attention with state-of-the-art performance on three benchmark datasets: Flickr8k, Flickr30k and MS COCO.

PaperPDFCodeCode Syntology ran

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, Syntology's MCP tool lists every function and class Syntology harvested from this paper and whether it ran (how to connect): get_harvested_code_for_paper(arxiv_id="1502.03044")

Code

Syntology Ran 15 of 84 code samples harvested from 22 repositories linked to this paper; 69 have no recorded run. Of those that ran: 3 ran · honoured contract; 10 ran · our draft was wrong; 1 ran · fixture could not drive it; 1 ran with no contract checked.

By repository: community (archive-listed): 84 samples from 22 repositories, 15 ran. The run record, sample by sample. “Ran” means executed on a synthesized input, not that the code is correct or reproduces the paper.

92 repositories listed; official and paper-mentioned ones first.

28-08-2000/ImageCaptioning mentioned on GitHubpytorch report
Aaatresh/Seeing-through-sound mentioned on GitHubMIT report
AaronCCWong/Show-Attend-and-Tell mentioned on GitHubpytorch report
AndreiMoraru123/ContextCollector mentioned on GitHubpytorchMIT report
Anjaney1999/image-captioning-seqgan mentioned on GitHubpytorch report
AtluriNikhil/Image-Captioning mentioned on GitHubtf report
Danyache/skoltech_image_cap mentioned on GitHubpytorch report
DeepRNN/image_captioning mentioned on GitHubtfMIT report
Japanese-Image-Captioning/SAT-for-Japanese mentioned on GitHubpytorchMIT report
LaurentVe/Automatic-image-Captioning mentioned on GitHubpytorch report
LinXueyuanStdio/LaTeX_OCR mentioned on GitHubtfApache-2.0 report
LinXueyuanStdio/LaTeX_OCR_PRO mentioned on GitHubtfGPL-3.0 report
Mogbo/image-caption mentioned on GitHubtfMIT report
NicholasKX/ShowAttendTell mentioned on GitHubmindsporeApache-2.0 report
Pillercottrer/radcap_project mentioned on GitHubpytorch report
Sugarc0de/image_captioning mentioned on GitHubpytorch report
Surojit-KB/ARTY mentioned on GitHubtf report
TheZino/Image_captioning_SATell mentioned on GitHubpytorch report
ajamjoom/Image-Captions mentioned on GitHubpytorch report
am-sirdaniel/Image-Captioning-Model mentioned on GitHubpytorch report
anirbrhm/ImageCaptioning mentioned on GitHubpytorch report
arunadurai/Eye-For-Blind mentioned on GitHub report
atharv6/Image-Captioning mentioned on GitHubpytorch report
benpeloquin7/sapir_image_captioning mentioned on GitHubpytorch report
changji2069/Scope-Project mentioned on GitHub report
coldmanck/show-attend-and-tell mentioned on GitHubtfMIT report
danielajisafe/Image-Captioning-Model mentioned on GitHubpytorch report
feizc/Image-Caption-Baseline mentioned on GitHubpytorch report
feizc/Image-Caption-Pytorch mentioned on GitHubpytorch report
frgsimpson/kitt mentioned on GitHubtf report
fuqianya/show-attend-and-tell-paddle mentioned on GitHubpaddle report
gfoxx29/Image-Captioning mentioned on GitHubpytorchMIT report
glados97/dsa-model mentioned on GitHubpytorch report
hiyaroy12/multitask_learning mentioned on GitHubtf report
husnain-ali21/image_captioning mentioned on GitHubpytorch report
jaykshirsagar05/captionify mentioned on GitHub report
jianhanlim/ipr-imagecaptioning mentioned on GitHubtfBSD-3-Clause report
juletx/image-caption-generation mentioned on GitHubtfMIT report
kay-ck/bsc-attack mentioned on GitHubpytorch report
kelvinxu/arctic-captions mentioned on GitHub report
kenkai21/Image_Captioning mentioned on GitHubpytorch report
kenya-sk/show_attend_and_tell mentioned on GitHubpytorch report
khangt1k25/Image-Captioning mentioned on GitHubtf report
krasserm/fairseq-image-captioning mentioned on GitHubpytorch report
leob03/Image_captionning mentioned on GitHubpytorchMIT report
lvyufeng/image_captioning_mindspore mentioned on GitHubmindsporeMIT report
martincss/first-captioning mentioned on GitHubtfMIT report
maxy0524/image_captioning mentioned on GitHubtfMIT report
nalbert9/Image-Captioning mentioned on GitHubpytorchMIT report
nameoverflow/neuro-cangjie mentioned on GitHubpytorch report
nishimehta/Image_Captioning mentioned on GitHubtf report
phisad/keras-shatt mentioned on GitHubtfGPL-3.0 report
prasadsana/Eye_for_blind mentioned on GitHub report
radpet/fake-news-detector mentioned on GitHubtf report
riron1206/kaggle_MoA mentioned on GitHubpytorch report
sathvik-sankruth/Term-project mentioned on GitHubtf report
shaox192/NeuralGuidance mentioned on GitHubpytorchApache-2.0 report
smisthzhu/attentionocr mentioned on GitHubtf report
snagiri/ECE285_Jarvis_ProjectA mentioned on GitHubpytorch report
souvikshanku/show-attend-and-tell mentioned on GitHubpytorchMIT report
stingram/CV_Image_Captioning mentioned on GitHubpytorch report
xuewyang/Fashion_Captioning mentioned on GitHubpytorchNOASSERTION report
yunjey/show-attend-and-tell mentioned on GitHubtfMIT report
yurayli/image-caption-pytorch mentioned on GitHubpytorch report
yurayli/image_caption_pytorch mentioned on GitHubpytorchMIT report
zhang0jhon/AttentionOCR mentioned on GitHubtf report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

84 samples harvested; 15 ran; 3 honoured the contract we drafted; 69 have no recorded run. Read from Syntology's graph 2026-09-24; that is when this build read the record, not when the samples ran.

3ran · honoured contract
10ran · our draft was wrong
1ran · fixture could not drive it
1ran
69unverified

Licence: 7 of the 84 samples are pointer only, meaning Syntology does not serve that copy's text. This page shows no code text for any sample; each one links to its file in the repository.

Harvested from 22 repositories linked to this paper, official or community; each sample names its own and says which. “Ran” means the sample executed on a synthesized input. It does not mean the output is correct, and nothing here reproduces the paper's results. “Honoured” and “violated” refer to a contract Syntology drafted from the code itself; “our draft was wrong” and “fixture could not drive it” are failures of Syntology's instrument, not of the code.

Each sample ends with its code_sha256, Syntology's identity for that exact code. An agent fetches the stored sample with Syntology's MCP tool get_code(code_sha256="…") (how to connect); click an identity to copy that call.

Repository labels, per sample. official repository: The archive marks this repository official for the paper. named in the paper: The archive records that the paper mentions this repository; it is not marked official. community (archive-listed): In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper. found in paper text by Syntology: Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted. community: Not in the archive's code links for this paper; a community repository Syntology harvested. Samples from a repository marked official are listed first. Licence labels name the repository's licence as recorded at harvest. “Pointer only” means Syntology does not serve that copy's text, for one of four reasons: no licence file was found; the licence was not identified; the licence is recorded as permissive but that copy's record is not marked cleared; or the licence is outside the permissive list Syntology serves text under (MIT, Apache-2.0, BSD and similar). Some licences outside that list permit redistribution, such as WTFPL, and GPL-3.0 under its conditions; they are simply not on the list. Hover a licence label for the reason. File links open the file on GitHub at the default branch, which may have changed since the harvest.

accuracy shaox192/NeuralGuidance/analysis/utils.py community (archive-listed) ran · fixture could not drive it Apache-2.0 (permissive) · b0f936d4d6ae3b8c · report
accuracy Japanese-Image-Captioning/SAT-for-Japanese/utils.py community (archive-listed) ran MIT (permissive) · 53daad1d7b99fbdf · report
calc_max_length ishritam/Image-captioning-with-visual-attention/image_cap.py community (archive-listed) ran · honoured contract no licence file found · pointer only · af5b85578e4e6f1b · report
cook_refs coldmanck/show-attend-and-tell/utils/coco/pycocoevalcap/cider/cider_scorer.py community (archive-listed) ran · our draft was wrong MIT (permissive) · 4c1b107103b7c856 · report
cook_test coldmanck/show-attend-and-tell/utils/coco/pycocoevalcap/cider/cider_scorer.py community (archive-listed) ran · our draft was wrong fingerprinted MIT (permissive) · 3573c4bc85f65dfd · report
create_padding_mask krasserm/fairseq-image-captioning/model/caption.py community (archive-listed) ran · our draft was wrong Apache-2.0 (permissive) · f93be7ac25d7dac1 · report
itemlist Zhangyuewei98/arctic-captions-py3/capgen.py community (archive-listed) ran · our draft was wrong no licence file found · pointer only · 8569db484e9d39b6 · report
load_textfiles Zhangyuewei98/arctic-captions-py3/metrics.py community (archive-listed) ran · our draft was wrong no licence file found · pointer only · 05ec0ad6e8114d01 · report
pil_loader shaox192/NeuralGuidance/manifold_guidance/man_data_loader.py community (archive-listed) ran · honoured contract Apache-2.0 (permissive) · f6be833ba409926f · report
pil_loader djain454/Show-Attend-and-Tell-Neural-Image-Caption-Generation-with-Visual-Attention/dataset.py community (archive-listed) ran · honoured contract Apache-2.0 (permissive) · f321f54723433661 · report
precook coldmanck/show-attend-and-tell/utils/coco/pycocoevalcap/bleu/bleu_scorer.py community (archive-listed) ran · our draft was wrong fingerprinted MIT (permissive) · ec4448373d72d636 · report
precook coldmanck/show-attend-and-tell/utils/coco/pycocoevalcap/cider/cider_scorer.py community (archive-listed) ran · our draft was wrong fingerprinted MIT (permissive) · 9560e3e3e41bc1b2 · report
remove_special_tokens yurayli/image_caption_pytorch/eval_metrics.py community (archive-listed) ran · our draft was wrong fingerprinted MIT (permissive) · 9bfeeae84da4c830 · report
to_var TheZino/Image_captioning_SATell/models.py community (archive-listed) ran · our draft was wrong fingerprinted no licence file found · pointer only · 502c50812b7f31ab · report
word_for_id jaykshirsagar05/captionify/fastapi/caption.py community (archive-listed) ran · our draft was wrong MIT (permissive) · 8a04d895237560a5 · report
BLEU_score yurayli/image_caption_pytorch/eval_metrics.py community (archive-listed) unverified MIT (permissive) · 4e8ebb8599fb73e6 · report
Magnitude shaox192/NeuralGuidance/analysis/smoothness.py community (archive-listed) unverified Apache-2.0 (permissive) · 88947ef0639ed6bb · report
accuracy NicholasKX/ShowAttendTell/utils.py community (archive-listed) unverified Apache-2.0 (permissive) · 95216f51978817ce · report
accuracy djain454/Show-Attend-and-Tell-Neural-Image-Caption-Generation-with-Visual-Attention/utils.py community (archive-listed) unverified Apache-2.0 (permissive) · fc7e660e3ad2dd0e · report
attention_visualizer leob03/Image_captionning/utils/utils.py community (archive-listed) unverified MIT (permissive) · 754d759415cbca01 · report
batchify yurayli/image_caption_pytorch/encode_image.py community (archive-listed) unverified MIT (permissive) · a454bc2657b5d55b · report
cal_noise_ceiling_single_vox shaox192/NeuralGuidance/analysis/noise_ceiling.py community (archive-listed) unverified Apache-2.0 (permissive) · ecb092757c4db425 · report
calculate_caption_lengths djain454/Show-Attend-and-Tell-Neural-Image-Caption-Generation-with-Visual-Attention/utils.py community (archive-listed) unverified Apache-2.0 (permissive) · 9f843b78744f8406 · report
caption_features martincss/first-captioning/preprocess_tokenize_captions.py community (archive-listed) unverified MIT (permissive) · 68d2533729c9efa1 · report
caption_image_beam_search Japanese-Image-Captioning/SAT-for-Japanese/caption.py community (archive-listed) unverified MIT (permissive) · 08e797d6dcd0b472 · report
cifar10 leob03/Image_captionning/utils/data.py community (archive-listed) unverified MIT (permissive) · ffff69e68ad972d9 · report
compute_numeric_gradient leob03/Image_captionning/utils/grad.py community (archive-listed) unverified MIT (permissive) · 112c9a161e488e7c · report
cook_refs DeepRNN/image_captioning/utils/coco/pycocoevalcap/cider/cider_scorer.py community (archive-listed) unverified MIT (permissive) · 5172aba957a32d9e · report
cook_refs coldmanck/show-attend-and-tell/utils/coco/pycocoevalcap/bleu/bleu_scorer.py community (archive-listed) unverified MIT (permissive) · 7586e1d32f7325f7 · report
cook_refs maxy0524/image_captioning/utils/coco/pycocoevalcap/bleu/bleu_scorer.py community (archive-listed) unverified MIT (permissive) · ce924695dcab9465 · report
cook_test DeepRNN/image_captioning/utils/coco/pycocoevalcap/cider/cider_scorer.py community (archive-listed) unverified MIT (permissive) · 90a4bcbf05586ff3 · report
cook_test coldmanck/show-attend-and-tell/utils/coco/pycocoevalcap/bleu/bleu_scorer.py community (archive-listed) unverified MIT (permissive) · 99e54843980299e2 · report
cook_test maxy0524/image_captioning/utils/coco/pycocoevalcap/bleu/bleu_scorer.py community (archive-listed) unverified MIT (permissive) · 2ac8c35153e01ef8 · report
correlation shaox192/NeuralGuidance/neural_guidance_training/model.py community (archive-listed) unverified Apache-2.0 (permissive) · e9b0f77f1b690e9a · report
create_emb Skumarr53/Image-Caption-Generation-using-Fastai/modules/model.py community (archive-listed) unverified no licence file found · pointer only · 066b8316b935d089 · report
data_gen Aaatresh/Seeing-through-sound/vOICe/data_generators/data_generator_cifar10.py community (archive-listed) unverified MIT (permissive) · 48f167eac6c6fdf1 · report
decode_captions yurayli/image_caption_pytorch/data_utils.py community (archive-listed) unverified MIT (permissive) · b37f9c21d668e58f · report
extract_features yurayli/image_caption_pytorch/solver.py community (archive-listed) unverified MIT (permissive) · 92cdbcd5d83f0f6d · report
feed_forward_net yurayli/image_caption_pytorch/encode_image.py community (archive-listed) unverified MIT (permissive) · aa4791e2bffdf413 · report
getFileNameToSave LinXueyuanStdio/LaTeX_OCR/visualize_attention.py community (archive-listed) unverified Apache-2.0 (permissive) · 891b8f71cfafca9a · report
getOutArray LinXueyuanStdio/LaTeX_OCR/visualize_attention.py community (archive-listed) unverified Apache-2.0 (permissive) · d7419b2156f83b93 · report
getWH LinXueyuanStdio/LaTeX_OCR/visualize_attention.py community (archive-listed) unverified Apache-2.0 (permissive) · 54e482c9c86eec94 · report
get_dist_matrix shaox192/NeuralGuidance/analysis/RSA.py community (archive-listed) unverified Apache-2.0 (permissive) · 8a9b2d5877d086a3 · report
get_embeddings LinXueyuanStdio/LaTeX_OCR/model/decoder.py community (archive-listed) unverified Apache-2.0 (permissive) · 9b24ffeda92bc6e1 · report
get_hypotheses lvyufeng/image_captioning_mindspore/src/utils.py community (archive-listed) unverified MIT (permissive) · 2d423582a2c44126 · report
get_model_corr_mat shaox192/NeuralGuidance/analysis/RSA.py community (archive-listed) unverified Apache-2.0 (permissive) · 70e878dc080d6418 · report
get_model_weights shaox192/NeuralGuidance/manifold_guidance/man_cotrain.py community (archive-listed) unverified Apache-2.0 (permissive) · 29c867d79b8c8f8e · report
get_references lvyufeng/image_captioning_mindspore/src/utils.py community (archive-listed) unverified MIT (permissive) · 9ee8094f76ff694d · report
get_repeated_image shaox192/NeuralGuidance/analysis/noise_ceiling.py community (archive-listed) unverified Apache-2.0 (permissive) · cd8511add99a8df2 · report
get_toy_data leob03/Image_captionning/main_func.py community (archive-listed) unverified MIT (permissive) · c5e0eb463e9afc32 · report
image_features_extracter martincss/first-captioning/preprocess_encode_images.py community (archive-listed) unverified MIT (permissive) · 02fa9793705ee294 · report
image_fnames_captions martincss/first-captioning/coco_utils.py community (archive-listed) unverified MIT (permissive) · c369482485646bd1 · report
instantiate_ROI_man_model shaox192/NeuralGuidance/manifold_guidance/man_models.py community (archive-listed) unverified Apache-2.0 (permissive) · 9a5a579534afd15e · report
load codekansas/keras-language-modeling/generate_insurance_qa_embeddings.py community (archive-listed) unverified MIT (permissive) · eccb788cbbf8f688 · report
load_checkpoint fuqichen1998/eecs442-final-project-show-and-tell/image-caption/helper.py community (archive-listed) unverified MIT (permissive) · 83e927f706f3e92d · report
load_coco_captions leob03/Image_captionning/main_func.py community (archive-listed) unverified MIT (permissive) · 088be7e4343bda87 · report
load_data_folder shaox192/NeuralGuidance/manifold_guidance/man_data_loader.py community (archive-listed) unverified Apache-2.0 (permissive) · 15f22915a39fba42 · report
load_embeddings Japanese-Image-Captioning/SAT-for-Japanese/utils.py community (archive-listed) unverified MIT (permissive) · 4cc9097b30b644e0 · report
load_from_nii shaox192/NeuralGuidance/analysis/utils.py community (archive-listed) unverified Apache-2.0 (permissive) · c2c62f3f2bdbc43c · report
load_image martincss/first-captioning/preprocess_encode_images.py community (archive-listed) unverified MIT (permissive) · b0bb3778f118bff1 · report
load_img_folder_ls shaox192/NeuralGuidance/manifold_guidance/man_data_loader.py community (archive-listed) unverified Apache-2.0 (permissive) · b8d0de38fa1b3c8b · report
make_tokenizer martincss/first-captioning/preprocess_tokenize_captions.py community (archive-listed) unverified MIT (permissive) · 6954f2198f23dbee · report
pickle_load shaox192/NeuralGuidance/analysis/utils.py community (archive-listed) unverified Apache-2.0 (permissive) · 198d90979fd0346e · report
precook DeepRNN/image_captioning/utils/coco/pycocoevalcap/cider/cider_scorer.py community (archive-listed) unverified MIT (permissive) · 422c41f5b22cf9c3 · report
precook maxy0524/image_captioning/utils/coco/pycocoevalcap/bleu/bleu_scorer.py community (archive-listed) unverified MIT (permissive) · 1f8818c755aa7d9a · report
preprocess yurayli/image_caption_pytorch/encode_image.py community (archive-listed) unverified MIT (permissive) · 900ff9988e479210 · report
process_caption_tokens djain454/Show-Attend-and-Tell-Neural-Image-Caption-Generation-with-Visual-Attention/generate_json_data.py community (archive-listed) unverified Apache-2.0 (permissive) · 369a98c031b8ad9d · report
rel_error leob03/Image_captionning/utils/grad.py community (archive-listed) unverified MIT (permissive) · 0a60ca96331ba158 · report
rnn_forward leob03/Image_captionning/rnn_lstm_captioning.py community (archive-listed) unverified MIT (permissive) · 818e7744734a9021 · report
rnn_step_backward leob03/Image_captionning/rnn_lstm_captioning.py community (archive-listed) unverified MIT (permissive) · 7459c63f19bd1a20 · report
rnn_step_forward leob03/Image_captionning/rnn_lstm_captioning.py community (archive-listed) unverified MIT (permissive) · a4639ecce23676b1 · report
run_clean shaox192/NeuralGuidance/analysis/RSA.py community (archive-listed) unverified Apache-2.0 (permissive) · e2a84722a01bf8a0 · report
sample_from_start Anjaney1999/image-captioning-seqgan/pretrain_discriminator.py community (archive-listed) unverified no licence file found · pointer only · 22225c242e4c65f6 · report
search_caption fuqichen1998/eecs442-final-project-show-and-tell/image-caption/visualize.py community (archive-listed) unverified MIT (permissive) · f6f129c245afaefa · report
split_captions yurayli/image_caption_pytorch/data_utils.py community (archive-listed) unverified MIT (permissive) · 5ce371a11110cca1 · report
split_hparams martincss/first-captioning/grid_search.py community (archive-listed) unverified MIT (permissive) · 8b4a39f185ba405c · report
split_image_files yurayli/image_caption_pytorch/data_utils.py community (archive-listed) unverified MIT (permissive) · b6424d334cb20b0b · report
tensor_to_image leob03/Image_captionning/utils/utils.py community (archive-listed) unverified MIT (permissive) · bc260615679a81dd · report
tokenize yurayli/image_caption_pytorch/tokenize_caption.py community (archive-listed) unverified MIT (permissive) · d85980a56655da31 · report
train_captioner leob03/Image_captionning/main_func.py community (archive-listed) unverified MIT (permissive) · 0a5ea4832aa6d5f3 · report
unpickle Aaatresh/Seeing-through-sound/vOICe/data_generators/data_generator_cifar10.py community (archive-listed) unverified MIT (permissive) · bfc501c46e8ad1f9 · report
unzip Zhangyuewei98/arctic-captions-py3/capgen.py community (archive-listed) unverified no licence file found · pointer only · 73d1b90c1f18304d · report
visualize_dataset leob03/Image_captionning/utils/utils.py community (archive-listed) unverified MIT (permissive) · f534fbe3633b3692 · report
word_idx_map yurayli/image_caption_pytorch/tokenize_caption.py community (archive-listed) unverified MIT (permissive) · 6c6770018fc47bec · report

Tasks

Caption GenerationImage CaptioningTranslation

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections