{"url":"/dataset/mnumgluesub","name":"MNumGLUESub","full_name":null,"description_markdown":"The **MNumGLUESub** dataset is a **multi-task arithmetic reasoning benchmark**. It is part of the **NumGLUE** collection, which focuses on evaluating mathematical reasoning abilities in various languages. Let's break down the details:\r\n\r\n1. **NumGLUE**: This broader benchmark suite includes several tasks related to mathematical reasoning. One of its subsets is **MNumGLUESub**.\r\n\r\n2. **MNumGLUESub**:\r\n    - **Tasks Selection**: To create this dataset, specific tasks from the larger NumGLUE collection were chosen. Specifically, tasks 1, 4, and 8 were selected.\r\n    - **Mathematical World Problems**: These tasks involve solving mathematical world problems. The questions in MNumGLUESub cover various mathematical concepts and require reasoning skills.\r\n    - **Translation**: The questions from MNumGLUESub were translated into **nine different languages** using ChatGPT. These languages align with the languages used in the broader GSM8KInstruct dataset.\r\n\r\n3. **Purpose**: MNumGLUESub serves as an evaluation dataset for assessing the mathematical reasoning capabilities of models across different languages. It helps researchers analyze how well models perform on arithmetic reasoning tasks in multilingual settings.\r\n\r\nFor further details, you can refer to the research paper on **Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimization (MAPO)**, which introduces the MAPO framework for enhancing multilingual reasoning abilities ¹².\r\n\r\nSource: Conversation with Bing, 3/17/2024\r\n(1) MAPO: Advancing Multilingual Reasoning through Multilingual-Alignment .... https://arxiv.org/html/2401.06838v2.\r\n(2) MAPO: Advancing Multilingual Reasoning through Multilingual Alignment .... https://arxiv.org/html/2401.06838v1.\r\n(3) [2401.06838] MAPO: Advancing Multilingual Reasoning through .... https://arxiv.org/abs/2401.06838.","description_withheld":null,"homepage":"","introduced_date":"2024-01-12","introduced_date_note":null,"introduced_by":{"paper":"/paper/mapo-advancing-multilingual-reasoning-through","title":"MAPO: Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimization","first_author":"Shuaijie She","url":null},"license":null,"modalities":[],"tasks":[],"languages":[],"variants":["MNumGLUESub"],"data_loaders":[],"num_papers_in_archive":1,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[],"papers_with_a_benchmark_row":[],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}