{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/vistext-a-benchmark-for-semantically-rich","title":"VisText: A Benchmark for Semantically Rich Chart Captioning","arxiv_id":"2307.05356","date":"2023-06-28","proceeding":null,"authors":["Benny J. Tang","Angie Boggust","Arvind Satyanarayan"],"abstract":"Captions that describe or explain charts help improve recall and comprehension of the depicted data and provide a more accessible medium for people with visual disabilities. However, current approaches for automatically generating such captions struggle to articulate the perceptual or cognitive features that are the hallmark of charts (e.g., complex trends and patterns). In response, we introduce VisText: a dataset of 12,441 pairs of charts and captions that describe the charts' construction, report key statistics, and identify perceptual and cognitive phenomena. In VisText, a chart is available as three representations: a rasterized image, a backing data table, and a scene graph -- a hierarchical representation of a chart's visual elements akin to a web page's Document Object Model (DOM). To evaluate the impact of VisText, we fine-tune state-of-the-art language models on our chart captioning task and apply prefix-tuning to produce captions that vary the semantic content they convey. Our models generate coherent, semantically rich captions and perform on par with state-of-the-art chart captioning models across machine translation and text generation metrics. Through qualitative analysis, we identify six broad categories of errors that our models make that can inform future work.","url_abs":"https://arxiv.org/abs/2307.05356v1","url_pdf":"https://arxiv.org/pdf/2307.05356v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"vistext-a-benchmark-for-semantically-rich","repo_url":"https://github.com/mitvis/vistext","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"GPL-3.0"}}],"tasks":[{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"text-generation","task_name":"Text Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2307.05356","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.05356"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mitvis/vistext","reach":{"status":"ok","spdx":"GPL-3.0"}}],"summary":{"ran":10,"unverified":1},"by_repo_kind":{"official":{"samples":11,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":11,"samples":[{"code_sha256_prefix":"41c46b0d5713eb14","entry":"all_gather","repo":"mitvis/vistext","repo_kind":"official","path":"code/image_guided/dist_utils.py","file_url":"https://github.com/mitvis/vistext/blob/HEAD/code/image_guided/dist_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"41c46b0d5713eb14"}},{"code_sha256_prefix":"c11c62760530db91","entry":"cleanL2L3","repo":"mitvis/vistext","repo_kind":"official","path":"code/utils_gen.py","file_url":"https://github.com/mitvis/vistext/blob/HEAD/code/utils_gen.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"c11c62760530db91"}},{"code_sha256_prefix":"855485cd9c595a15","entry":"evalRG","repo":"mitvis/vistext","repo_kind":"official","path":"code/utils_metrics.py","file_url":"https://github.com/mitvis/vistext/blob/HEAD/code/utils_metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"855485cd9c595a15"}},{"code_sha256_prefix":"7dfe74447fc98501","entry":"evalRG_single","repo":"mitvis/vistext","repo_kind":"official","path":"code/utils_metrics.py","file_url":"https://github.com/mitvis/vistext/blob/HEAD/code/utils_metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"7dfe74447fc98501"}},{"code_sha256_prefix":"a0816452bef935d2","entry":"generate_title","repo":"mitvis/vistext","repo_kind":"official","path":"code/utils_gen.py","file_url":"https://github.com/mitvis/vistext/blob/HEAD/code/utils_gen.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"a0816452bef935d2"}},{"code_sha256_prefix":"30b5edaf5d8dee25","entry":"get_optimizer","repo":"mitvis/vistext","repo_kind":"official","path":"code/image_guided/param.py","file_url":"https://github.com/mitvis/vistext/blob/HEAD/code/image_guided/param.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"30b5edaf5d8dee25"}},{"code_sha256_prefix":"895e1f65c2bdbed9","entry":"parse_args","repo":"mitvis/vistext","repo_kind":"official","path":"code/image_guided/param.py","file_url":"https://github.com/mitvis/vistext/blob/HEAD/code/image_guided/param.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"895e1f65c2bdbed9"}},{"code_sha256_prefix":"55ae29a3f85c0bd1","entry":"str2bool","repo":"mitvis/vistext","repo_kind":"official","path":"code/image_guided/param.py","file_url":"https://github.com/mitvis/vistext/blob/HEAD/code/image_guided/param.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"55ae29a3f85c0bd1"}},{"code_sha256_prefix":"a83a7eb15f404822","entry":"tokenizerEncDec","repo":"mitvis/vistext","repo_kind":"official","path":"code/utils_metrics.py","file_url":"https://github.com/mitvis/vistext/blob/HEAD/code/utils_metrics.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"a83a7eb15f404822"}},{"code_sha256_prefix":"33b1450e6900b7f4","entry":"wordsub","repo":"mitvis/vistext","repo_kind":"official","path":"code/utils_gen.py","file_url":"https://github.com/mitvis/vistext/blob/HEAD/code/utils_gen.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"33b1450e6900b7f4"}},{"code_sha256_prefix":"c8c2e5c9597c3212","entry":"convert_slow_vlt5tokenizer","repo":"mitvis/vistext","repo_kind":"official","path":"code/image_guided/tokenization.py","file_url":"https://github.com/mitvis/vistext/blob/HEAD/code/image_guided/tokenization.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"GPL-3.0","inline_ok":false,"mcp_get_code":{"code_sha256":"c8c2e5c9597c3212"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}