{"url":"/dataset/mmt-bench","name":"MMT-Bench","full_name":null,"description_markdown":"MMT-Bench is a comprehensive benchmark designed to evaluate Large Vision-Language Models (LVLMs) across a wide array of multimodal tasks that require expert knowledge as well as deliberate visual recognition, localization, reasoning, and planning¹. It includes **31,325 meticulously curated multi-choice visual questions** from various scenarios such as vehicle driving and embodied navigation, covering **32 core meta-tasks** and **162 subtasks** in multimodal understanding¹.\r\n\r\nThe benchmark aims to be comprehensive enough to assess the multitask performance of LVLMs using a task map, which helps in identifying both in-domain and out-of-domain tasks. This extensive coverage allows for a thorough evaluation of the models' capabilities in multimodal understanding¹.\r\n\r\n(1) MMT-Bench. https://mmt-bench.github.io/.\r\n(2) OpenGVLab/MMT-Bench: ICML'2024 - GitHub. https://github.com/OpenGVLab/MMT-Bench.\r\n(3) OpenGVLab/MMT-Bench - Giters. https://giters.com/OpenGVLab/MMT-Bench.","description_withheld":null,"homepage":"https://github.com/OpenGVLab/MMT-Bench","introduced_date":"2024-04-24","introduced_date_note":null,"introduced_by":{"paper":"/paper/mmt-bench-a-comprehensive-multimodal","title":"MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI","first_author":"Kaining Ying","url":null},"license":null,"modalities":[],"tasks":[],"languages":[],"variants":["MMT-Bench"],"data_loaders":[],"num_papers_in_archive":21,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[],"papers_with_a_benchmark_row":[],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}