QuRe

Introduction

Generalized quantifiers (e.g., few, most) are used to indicate the proportions predicates are satisfied. QuRe is quantifier reasoning dataset from Pragmatic Reasoning Unlocks Quantifier Semantics for Foundation Models. It includes real-world sentences from Wikipedia and human annotations of generalized quantifiers from English speakers.

Sample

{
    "orig_sentence": "In order for a steel to be considered stainless it must have a Chromium content of at least 10.5%.", 
    "percentage": "10.50%", 
    "percentage_index": 0, 
    "math_expr": ">=0.105", 
    "quant_sent": "In order for a steel to be considered stainless it must have some Chromium content.", 
    "quantifier": "some", 
    "quantifier_position": 12, 
    "specificity": "unable", 
    "wiki_entity": "List of blade materials", 
    "topics": "metallurgy; steel; composition"
}

• orig_sentence: the original sentence appeared in Wikipedia.
• percentage: the percentage mentioned in the orig_sentence.
• percentage_index: the index of the mentioned percentage in the orig_sentence.
• math_expr: the percentage expression generated.
• quant_sent: the annotated quantified sentence.
• quantifier_position: the position of quantifier mentioned.
• specificity: the difficulty of deciphering the percentage scope of the quantifier from the sentence excluding the quantifier.
• wiki_entity: the wikipedia entity that includes orig_sentence in the wikipage content.
• topics: sentence topics.

Load Dataset

from datasets import load_dataset

ds = load_dataset("billli/QuRe")

Reference

@inproceedings{li-etal-2023-pragmatic,
    title = "Pragmatic Reasoning Unlocks Quantifier Semantics for Foundation Models",
    author = "Li, Yiyuan  and
      Menon, Rakesh  and
      Ghosh, Sayan  and
      Srivastava, Shashank",
    editor = "Bouamor, Houda  and
      Pino, Juan  and
      Bali, Kalika",
    booktitle = "Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing",
    month = dec,
    year = "2023",
    address = "Singapore",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.emnlp-main.38",
    pages = "573--591",
}