Papers › Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks
Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks
Jiasen Lu, Christopher Clark, Rowan Zellers, Roozbeh Mottaghi, Aniruddha Kembhavi
We propose Unified-IO, a model that performs a large variety of AI tasks spanning classical computer vision tasks, including pose estimation, object detection, depth estimation and image generation, vision-and-language tasks such as region captioning and referring expression, to natural language processing tasks such as question answering and paraphrasing. Developing a single unified model for such a large variety of tasks poses unique challenges due to the heterogeneous inputs and outputs pertaining to each task, including RGB images, per-pixel maps, binary masks, bounding boxes, and language. We achieve this unification by homogenizing every supported input and output into a sequence of discrete vocabulary tokens. This common representation across all tasks allows us to train a single transformer-based architecture, jointly on over 90 diverse datasets in the vision and language fields. Unified-IO is the first model capable of performing all 7 tasks on the GRIT benchmark and produces strong results across 16 diverse benchmarks like NYUv2-Depth, ImageNet, VQA2.0, OK-VQA, Swig, VizWizGround, BoolQ, and SciTail, with no task-specific fine-tuning. Code and demos for Unified-IO are available at: https://unified-io.allenai.org.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Object Categorization | GRIT | Unified-IOXL | Categorization (ablation) | 61.7 | #1 of 4 | Archive leaderboard | report |
| Object Categorization | GRIT | Unified-IOXL | Categorization (test) | 60.8 | #1 of 4 | Archive leaderboard | report |
| Object Localization | GRIT | Unified-IOXL | Localization (ablation) | 67.0 | #1 of 3 | Archive leaderboard | report |
| Object Localization | GRIT | Unified-IOXL | Localization (test) | 67.1 | #1 of 3 | Archive leaderboard | report |
| Object Segmentation | GRIT | Unified-IOXL | Segmentation (ablation) | 56.3 | #1 of 2 | Archive leaderboard | report |
| Object Segmentation | GRIT | Unified-IOXL | Segmentation (test) | 56.5 | #1 of 2 | Archive leaderboard | report |
| Visual Question Answering (VQA) | GRIT | Unified-IOXL | VQA (ablation) | 74.5 | #1 of 2 | Archive leaderboard | report |
| Visual Question Answering (VQA) | GRIT | Unified-IOXL | VQA (test) | 74.5 | #1 of 2 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections