Papers › SciCap: Generating Captions for Scientific Figures
SciCap: Generating Captions for Scientific Figures
Ting-Yao Hsu, C. Lee Giles, Ting-Hao 'Kenneth' Huang
Researchers use figures to communicate rich, complex information in scientific papers. The captions of these figures are critical to conveying effective messages. However, low-quality figure captions commonly occur in scientific articles and may decrease understanding. In this paper, we propose an end-to-end neural framework to automatically generate informative, high-quality captions for scientific figures. To this end, we introduce SCICAP, a large-scale figure-caption dataset based on computer science arXiv papers published between 2010 and 2020. After pre-processing - including figure-type classification, sub-figure identification, text normalization, and caption text selection - SCICAP contained more than two million figures extracted from over 290,000 papers. We then established baseline models that caption graph plots, the dominant (19.2%) figure type. The experimental results showed both opportunities and steep challenges of generating captions for scientific figures.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Datasets
Introduced by this paper, per the archive.
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Image Captioning | SCICAP | CNN+LSTM (Vision only, First sentence) | BLEU-4 | 0.0219 | #1 of 9 | Archive leaderboard | report |
| Image Captioning | SCICAP | CNN+LSTM (Text only, First sentence) | BLEU-4 | 0.0213 | #2 of 9 | Archive leaderboard | report |
| Image Captioning | SCICAP | CNN+LSTM (Text only, Single-Sent Caption) | BLEU-4 | 0.0212 | #3 of 9 | Archive leaderboard | report |
| Image Captioning | SCICAP | CNN+LSTM (Vision only, Single-Sent Caption) | BLEU-4 | 0.0207 | #4 of 9 | Archive leaderboard | report |
| Image Captioning | SCICAP | CNN+LSTM (Vision + Text, First sentence) | BLEU-4 | 0.0205 | #5 of 9 | Archive leaderboard | report |
| Image Captioning | SCICAP | CNN+LSTM (Vision + Text, Single-Sent Caption) | BLEU-4 | 0.0202 | #6 of 9 | Archive leaderboard | report |
| Image Captioning | SCICAP | CNN+LSTM (Vision only, Caption w/ <=100 words) | BLEU-4 | 0.0172 | #7 of 9 | Archive leaderboard | report |
| Image Captioning | SCICAP | CNN+LSTM (Vision + Text, Caption w/ <=100 words) | BLEU-4 | 0.0168 | #8 of 9 | Archive leaderboard | report |
| Image Captioning | SCICAP | CNN+LSTM (Text only, Caption w/ <=100 words) | BLEU-4 | 0.0165 | #9 of 9 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections