{"url":"/dataset/oab-exams","name":"OAB Exams","full_name":null,"description_markdown":"The **OAB Exams dataset** is a valuable resource used in the context of legal information systems. In Brazil, all legal professionals must demonstrate their knowledge of the law and its application by passing the **OAB exams**, which are the national bar exams. These exams serve as a benchmark for evaluating the performance of legal information systems. If a system can achieve a level of legal reasoning comparable to that of a human lawyer who successfully passes the OAB exam, it indicates significant progress.\r\n\r\nHere are some key details about the **OAB Exams dataset**:\r\n\r\n1. **Purpose**: The dataset is designed to address the problem of finding justifications for the answers to legal questions posed in the OAB exams.\r\n2. **Construction**: Researchers have constructed this dataset to facilitate experiments related to legal reasoning. It provides a foundation for evaluating the performance of legal information systems.\r\n3. **Baseline Performance**: Preliminary experiments using this dataset have established a baseline performance measure. Future improvements can be evaluated against this baseline.\r\n4. **Challenges**: Despite its importance, achieving high performance on this dataset remains challenging due to the complexity of legal reasoning.\r\n\r\nSource: Conversation with Bing, 3/16/2024\r\n(1) Passing the Brazilian OAB Exam: data preparation and some experiments. https://paperswithcode.com/paper/passing-the-brazilian-oab-exam-data.\r\n(2) eduagarcia/oab_exams at main - Hugging Face. https://huggingface.co/datasets/eduagarcia/oab_exams/tree/main.\r\n(3) eduagarcia-temp/llm_pt_leaderboard_requests · Datasets at Hugging Face. https://huggingface.co/datasets/eduagarcia-temp/llm_pt_leaderboard_requests.","description_withheld":null,"homepage":"https://huggingface.co/datasets/eduagarcia/oab_exams","introduced_date":"2017-12-14","introduced_date_note":null,"introduced_by":{"paper":"/paper/passing-the-brazilian-oab-exam-data","title":"Passing the Brazilian OAB Exam: data preparation and some experiments","first_author":"Pedro Delfino","url":null},"license":null,"modalities":[],"tasks":[{"name":"Text Generation","url":"/task/text-generation","datasets_with_task":"/datasets/task/text-generation"}],"languages":[],"variants":["OAB Exams"],"data_loaders":[],"num_papers_in_archive":3,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[{"leaderboard":"/sota/text-generation-on-oab-exams","task":"Text Generation","dataset_variant":"OAB Exams","rows":0,"metrics":["accuracy"],"first_row_in_archive_order":null,"note":"rows are the archive's own order at snapshot; nothing here re-ranks them"}],"papers_with_a_benchmark_row":[],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}