{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/reviewqa-a-relational-aspect-based-opinion","title":"ReviewQA: a relational aspect-based opinion reading dataset","arxiv_id":"1810.12196","date":"2018-10-29","proceeding":null,"authors":["Quentin Grail","Julien Perez"],"abstract":"Deep reading models for question-answering have demonstrated promising\nperformance over the last couple of years. However current systems tend to\nlearn how to cleverly extract a span of the source document, based on its\nsimilarity with the question, instead of seeking for the appropriate answer.\nIndeed, a reading machine should be able to detect relevant passages in a\ndocument regarding a question, but more importantly, it should be able to\nreason over the important pieces of the document in order to produce an answer\nwhen it is required. To motivate this purpose, we present ReviewQA, a\nquestion-answering dataset based on hotel reviews. The questions of this\ndataset are linked to a set of relational understanding competencies that we\nexpect a model to master. Indeed, each question comes with an associated type\nthat characterizes the required competency. With this framework, it is possible\nto benchmark the main families of models and to get an overview of what are the\nstrengths and the weaknesses of a given model on the set of tasks evaluated in\nthis dataset. Our corpus contains more than 500.000 questions in natural\nlanguage over 100.000 hotel reviews. Our setup is projective, the answer of a\nquestion does not need to be extracted from a document, like in most of the\nrecent datasets, but selected among a set of candidates that contains all the\npossible answers to the questions of the dataset. Finally, we present several\nbaselines over this dataset.","url_abs":"http://arxiv.org/abs/1810.12196v1","url_pdf":"http://arxiv.org/pdf/1810.12196v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"}],"methods":[],"datasets_introduced":[{"slug":"reviewqa","name":"ReviewQA","full_name":""}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}