SciGraphQA

SciGraphQA is a large-scale, open-domain dataset focused on generating multi-turn conversational question-answering dialogues centered around understanding and describing scientific graphs and figures. It contains over 300,000 samples derived from academic research papers in computer science and machine learning domains.

Each sample in ScFiGraphQA consists of a scientific graph image sourced from papers on ArXiv, accompanied by rich textual context including the paper's title, abstract, figure caption, and a paragraph from the paper referencing the figure. Using this comprehensive context, the dataset employs a to produce multi-turn question-answer dialogues aimed at explaining the given graph in an interactive, conversational format. On average, each sample contains 2-3 turns of question-answer exchange.

The key motivation behind SciGraphQA is providing a large-scale resource to support research and development of multi-modal AI systems that can engage in informative, open-ended conversations about graphs and data visualizations. The multi-turn dialogue format presents a more natural and interactive setting compared to standard visual question answering datasets that use fixed sets of standalone questions.

Potential use cases of SciGraphQA include pre-training and benchmarking multi-modal conversational models for scientific graph comprehension, building AI assistants that can discuss data insights, and developing aids to help individuals understand complex figures and diagrams interactively. The academic source material also provides a way to evaluate model capabilities on expert-level graphs spanning diverse topics and complex visual encodings.