{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/self-critical-sequence-training-for-image","title":"Self-critical Sequence Training for Image Captioning","arxiv_id":"1612.00563","date":"2016-12-02","proceeding":"CVPR 2017 7","authors":["Steven J. Rennie","Etienne Marcheret","Youssef Mroueh","Jarret Ross","Vaibhava Goel"],"abstract":"Recently it has been shown that policy-gradient methods for reinforcement\nlearning can be utilized to train deep end-to-end systems directly on\nnon-differentiable metrics for the task at hand. In this paper we consider the\nproblem of optimizing image captioning systems using reinforcement learning,\nand show that by carefully optimizing our systems using the test metrics of the\nMSCOCO task, significant gains in performance can be realized. Our systems are\nbuilt using a new optimization approach that we call self-critical sequence\ntraining (SCST). SCST is a form of the popular REINFORCE algorithm that, rather\nthan estimating a \"baseline\" to normalize the rewards and reduce variance,\nutilizes the output of its own test-time inference algorithm to normalize the\nrewards it experiences. Using this approach, estimating the reward signal (as\nactor-critic methods must do) and estimating normalization (as REINFORCE\nalgorithms typically do) is avoided, while at the same time harmonizing the\nmodel with respect to its test-time inference procedure. Empirically we find\nthat directly optimizing the CIDEr metric with SCST and greedy decoding at\ntest-time is highly effective. Our results on the MSCOCO evaluation sever\nestablish a new state-of-the-art on the task, improving the best result in\nterms of CIDEr from 104.9 to 114.7.","url_abs":"http://arxiv.org/abs/1612.00563v2","url_pdf":"http://arxiv.org/pdf/1612.00563v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/Dlut-lab-zmn/Image-Captioning-Attack","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/FJSam/SelfCritical_ImageCaptioning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/Kumar-Tarun/summarization","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/SJTU-VLG/ImageCaption-LBPF-qy","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/Zhendong-Wang/arsm_image_captioning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/arjung128/image-paragraph-captioning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/hjjpku/dynamic_graph_caption","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/husthuaan/AAT","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/husthuaan/AoANet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/jchenghu/sacreeos","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/jiahuei/COMIC-Compact-Image-Captioning-with-Attention","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/jiahuei/COMIC-Towards-A-Compact-Image-Captioning-Model-with-Attention","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/jiahuei/Self-Critical-SCST-TensorFlow","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/kangkang59812/GraphCaption","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/king-zark/self_critical","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/krasserm/fairseq-image-captioning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/lukemelas/image-paragraph-captioning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/mazm13/Image-to-Tree.pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/mykolesiko/nlp_projects","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/quantum-fusion/AI-deeplearning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/ruotianluo/GoogleConceptualCaptioning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/ruotianluo/ImageCaptioning.pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/ruotianluo/Transformer_Captioning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/ruotianluo/neuraltalk2.pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/ruotianluo/self-critical.pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/s-52/ImageCaptions","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/sgondala/GoogleConceptualCaptioning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/ukyh/switch_disc_caption","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/wtliao/ImageTransformer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/xiaobai714/image_caption","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"self-critical-sequence-training-for-image","repo_url":"https://github.com/ylqi/gl-rg","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-captioning","task_name":"Image Captioning"},{"task_slug":"policy-gradient-methods","task_name":"Policy Gradient Methods"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"},{"task_slug":"reinforcement-learning-2","task_name":"reinforcement-learning"}],"methods":[{"method_slug":"reinforce","method_name":"REINFORCE"},{"method_slug":"scst","method_name":"SCST"}],"datasets_introduced":[],"methods_introduced":[{"slug":"scst","name":"SCST","full_name":"Self-critical Sequence Training"}],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1612.00563","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1612.00563"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xiaobai714/image_caption","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Dlut-lab-zmn/Image-Captioning-Attack","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/husthuaan/AoANet","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/king-zark/self_critical","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ruotianluo/GoogleConceptualCaptioning","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ukyh/switch_disc_caption","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/hjjpku/dynamic_graph_caption","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kangkang59812/GraphCaption","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sgondala/GoogleConceptualCaptioning","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/SJTU-VLG/ImageCaption-LBPF-qy","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jiahuei/Self-Critical-SCST-TensorFlow","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mazm13/Image-to-Tree.pytorch","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/wtliao/ImageTransformer","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Zhendong-Wang/arsm_image_captioning","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/arjung128/image-paragraph-captioning","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ylqi/gl-rg","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/husthuaan/AAT","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/krasserm/fairseq-image-captioning","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jchenghu/sacreeos","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ruotianluo/Transformer_Captioning","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ruotianluo/self-critical.pytorch","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ruotianluo/ImageCaptioning.pytorch","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jiahuei/COMIC-Compact-Image-Captioning-with-Attention","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mykolesiko/nlp_projects","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/quantum-fusion/AI-deeplearning","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":7,"ran_violates":1,"unverified":5},"by_repo_kind":{"listed":{"samples":10,"ran":5,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"20bb0ff4a2d77a03","entry":"attention","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"20bb0ff4a2d77a03"}},{"code_sha256_prefix":"41b9ec0c269d8a91","entry":"build_vocab","repo":"ruotianluo/GoogleConceptualCaptioning","repo_kind":"listed","path":"scripts/prepro_labels.py","file_url":"https://github.com/ruotianluo/GoogleConceptualCaptioning/blob/HEAD/scripts/prepro_labels.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"41b9ec0c269d8a91"}},{"code_sha256_prefix":"a3722169bbc81569","entry":"clones","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"a3722169bbc81569"}},{"code_sha256_prefix":"4aad8242843ac47b","entry":"encode_captions","repo":"ruotianluo/GoogleConceptualCaptioning","repo_kind":"listed","path":"scripts/prepro_labels.py","file_url":"https://github.com/ruotianluo/GoogleConceptualCaptioning/blob/HEAD/scripts/prepro_labels.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4aad8242843ac47b"}},{"code_sha256_prefix":"d2379d710eedc4ae","entry":"pack_wrapper","repo":"husthuaan/AAT","repo_kind":"listed","path":"models/AttModel.py","file_url":"https://github.com/husthuaan/AAT/blob/HEAD/models/AttModel.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d2379d710eedc4ae"}},{"code_sha256_prefix":"bfac58a04b6835f5","entry":"pad_unsort_packed_sequence","repo":"husthuaan/AAT","repo_kind":"listed","path":"models/AttModel.py","file_url":"https://github.com/husthuaan/AAT/blob/HEAD/models/AttModel.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bfac58a04b6835f5"}},{"code_sha256_prefix":"e56d2f9cbd9cb8b1","entry":"sort_pack_padded_sequence","repo":"husthuaan/AAT","repo_kind":"listed","path":"models/AttModel.py","file_url":"https://github.com/husthuaan/AAT/blob/HEAD/models/AttModel.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e56d2f9cbd9cb8b1"}},{"code_sha256_prefix":"dd03765c7da9ca51","entry":"subsequent_mask","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"dd03765c7da9ca51"}},{"code_sha256_prefix":"0a4de8f498e3e70c","entry":"count_bad","repo":"husthuaan/AAT","repo_kind":"listed","path":"eval_utils.py","file_url":"https://github.com/husthuaan/AAT/blob/HEAD/eval_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0a4de8f498e3e70c"}},{"code_sha256_prefix":"06b72839dcd7433f","entry":"language_eval","repo":"husthuaan/AAT","repo_kind":"listed","path":"eval_utils.py","file_url":"https://github.com/husthuaan/AAT/blob/HEAD/eval_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"06b72839dcd7433f"}},{"code_sha256_prefix":"2b17380e9938d559","entry":"resnet18","repo":"husthuaan/AAT","repo_kind":"listed","path":"misc/resnet.py","file_url":"https://github.com/husthuaan/AAT/blob/HEAD/misc/resnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2b17380e9938d559"}},{"code_sha256_prefix":"d206d964368eeb24","entry":"resnet34","repo":"husthuaan/AAT","repo_kind":"listed","path":"misc/resnet.py","file_url":"https://github.com/husthuaan/AAT/blob/HEAD/misc/resnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d206d964368eeb24"}},{"code_sha256_prefix":"fade30a7dab7a039","entry":"resnet50","repo":"husthuaan/AAT","repo_kind":"listed","path":"misc/resnet.py","file_url":"https://github.com/husthuaan/AAT/blob/HEAD/misc/resnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fade30a7dab7a039"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}