{"url":"/dataset/bear-big","name":"BEAR-probe","full_name":"Benchmark for Evaluating Associative Reasoning","description_markdown":"The $\\text{BEAR}$ dataset and its larger version, $\\text{BEAR}_{\\text{big}}$, are benchmarks for evaluating common factual knowledge contained in language models. \r\n\r\nThis dataset was created as part of the [paper \"BEAR: A Unified Framework for Evaluating Relational Knowledge in Causal and Masked Language Models\"](https://arxiv.org/abs/2404.04113).\r\n\r\nFor more information visit the [LM Pub Quiz website](https://lm-pub-quiz.github.io/).\r\n\r\n## Citation\r\n\r\nWhen using the dataset or library, please cite the following paper:\r\n\r\n```\r\n@misc{wilandBEARUnifiedFramework2024,\r\n  title = {{{BEAR}}: {{A Unified Framework}} for {{Evaluating Relational Knowledge}} in {{Causal}} and {{Masked Language Models}}},\r\n  shorttitle = {{{BEAR}}},\r\n  author = {Wiland, Jacek and Ploner, Max and Akbik, Alan},\r\n  year = {2024},\r\n  number = {arXiv:2404.04113},\r\n  eprint = {2404.04113},\r\n  publisher = {arXiv},\r\n  url = {http://arxiv.org/abs/2404.04113},\r\n}\r\n```","description_withheld":null,"homepage":"https://lm-pub-quiz.github.io/","introduced_date":"2024-04-05","introduced_date_note":null,"introduced_by":{"paper":"/paper/bear-a-unified-framework-for-evaluating","title":"BEAR: A Unified Framework for Evaluating Relational Knowledge in Causal and Masked Language Models","first_author":"Jacek Wiland","url":null},"license":{"name":"CC BY-SA","url":null},"modalities":[{"name":"Texts","url":"/datasets/modality/texts"}],"tasks":[{"name":"General Knowledge","url":"/task/general-knowledge","datasets_with_task":"/datasets/task/general-knowledge"},{"name":"Knowledge Probing","url":"/task/knowledge-probing","datasets_with_task":"/datasets/task/knowledge-probing"},{"name":"Factual probe","url":"/task/factual-probe","datasets_with_task":"/datasets/task/factual-probe"},{"name":"Probing Language Models","url":"/task/probing-language-models","datasets_with_task":"/datasets/task/probing-language-models"},{"name":"World Knowledge","url":"/task/world-knowledge","datasets_with_task":"/datasets/task/world-knowledge"}],"languages":[{"name":"English","url":"/datasets/language/english"}],"variants":["BEAR-probe"],"data_loaders":[],"num_papers_in_archive":1,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[],"papers_with_a_benchmark_row":[],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}