{"url":"/dataset/m3exam","name":"M3Exam","full_name":null,"description_markdown":"**M3Exam** is a **multilingual, multimodal, and multilevel benchmark** designed for evaluating **Large Language Models (LLMs)**. Unlike traditional benchmarks, which often focus on specific tasks or datasets, M3Exam takes a more comprehensive approach by sourcing real and official **human exam questions**. Let's delve into its unique characteristics:\r\n\r\n1. **Multilingualism**: M3Exam encompasses questions from **multiple countries**, requiring strong multilingual proficiency and cultural knowledge. It evaluates how well LLMs handle diverse languages.\r\n\r\n2. **Multimodality**: Many exam questions are **multimodal**, combining text with images. M3Exam tests the model's ability to understand and process such complex, multimodal content.\r\n\r\n3. **Multilevel Structure**: M3Exam features exams from **three critical educational periods**, allowing a comprehensive assessment of a model's proficiency at different levels.\r\n\r\nHere are some key details about M3Exam:\r\n\r\n- **Number of Questions**: M3Exam contains **12,317 questions** in **9 diverse languages** across three educational levels.\r\n- **Image Processing**: Approximately **23%** of the questions require processing **images** for successful solving.\r\n\r\nDespite the existence of various benchmarks, M3Exam argues that **human exams** provide a more suitable means of evaluating general intelligence for large language models. These exams inherently demand a wide range of abilities, including **language understanding, domain knowledge, and problem-solving skills**.\r\n\r\nTop-performing LLMs, including **GPT-4**, have been assessed on M3Exam. However, they still face challenges with multilingual text, especially in **low-resource** and **non-Latin script languages**. Additionally, multimodal LLMs struggle with complex multimodal questions.\r\n\r\n(1) [2306.05179] M3Exam: A Multilingual, Multimodal, Multilevel Benchmark .... https://arxiv.org/abs/2306.05179.\r\n(2) M3Exam: A Multilingual, Multimodal, Multilevel Benchmark For Evaluating .... https://www.ai-summary.com/m3exam-a-multilingual-multimodal-multilevel-benchmark-for-evaluating-large-language-models/.\r\n(3) M3Exam: A Multilingual, Multimodal, Multilevel Benchmark for ... - DeepAI. https://deepai.org/publication/m3exam-a-multilingual-multimodal-multilevel-benchmark-for-examining-large-language-models.\r\n(4) M3Exam: A Multilingual , Multimodal , Multilevel ... - GitHub. https://github.com/DAMO-NLP-SG/M3Exam.\r\n(5) undefined. https://doi.org/10.48550/arXiv.2306.05179.","description_withheld":null,"homepage":"https://github.com/DAMO-NLP-SG/M3Exam","introduced_date":"2023-06-08","introduced_date_note":null,"introduced_by":{"paper":"/paper/m3exam-a-multilingual-multimodal-multilevel","title":"M3Exam: A Multilingual, Multimodal, Multilevel Benchmark for Examining Large Language Models","first_author":"Wenxuan Zhang","url":null},"license":null,"modalities":[],"tasks":[],"languages":[],"variants":["M3Exam"],"data_loaders":[],"num_papers_in_archive":32,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[],"papers_with_a_benchmark_row":[],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}