{"url":"/dataset/alta2021-evidencegrading","name":"ALTA 2021 Shared Task","full_name":"Automatic Grading of Evidence, 10 years later","description_markdown":"This dataset is described in the [ALTA 2021 Shared Task website](https://www.alta.asn.au/events/sharedtask2021/index.html) and associated [CodaLab competition](https://competitions.codalab.org/competitions/33739).\r\n\r\nThe basic task is to build an automatic evidence grading system for evidence-based medicine. Evidence-based medicine is a medical practice which requires practitioners to search medical literature for evidence when making clinical decisions. The practitioners are also required to grade the quality of extracted evidence on some chosen scale. The goal of the grading system is to automatically determine the grade of an evidence given the article abstract(s) from which the evidence is extracted.\r\n\r\nThe grading scale used for this task is the Strength of Recommendation Taxonomy (SORT). This taxonomy has 3 grades - A (strong), B (moderate) and C (weak). The grade of an evidence depends on multiple factors and information about this grading scale can be found in the paper by [Ebell et al. (2004)](https://www.jabfm.org/content/17/1/59).","description_withheld":null,"homepage":"https://www.alta.asn.au/events/sharedtask2021/index.html","introduced_date":"2021-12-01","introduced_date_note":null,"introduced_by":{"paper":"/paper/overview-of-the-2021-alta-shared-task","title":"Overview of the 2021 ALTA Shared Task: Automatic Grading of Evidence, 10 years later","first_author":"Diego Mollá","url":null},"license":null,"modalities":[{"name":"Texts","url":"/datasets/modality/texts"}],"tasks":[{"name":"Classification","url":"/task/classification-1","datasets_with_task":"/datasets/task/classification-1"}],"languages":[{"name":"English","url":"/datasets/language/english"}],"variants":["ALTA 2021 Shared Task"],"data_loaders":[],"num_papers_in_archive":4,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[],"papers_with_a_benchmark_row":[],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}