{"url":"/dataset/rare","name":"RARE","full_name":"Randomized AMRs with Rewired Edges","description_markdown":"RARE consists of English AMR pairs with similarity scores that reflect the structural differences between them.\r\n\r\nGiven that AMRs are graphical representations of text, an AMR similarity metric should be sensitive to structural variations between AMRs, even if its labels remain unchanged. Since there is no established evaluation of AMR metrics on structural similarity, we have developed a new benchmark dataset called Randomized AMRs with Rewired Edges (RARE). RARE consists of English AMR pairs with similarity scores that reflect the structural differences between them.\r\n\r\nIt can be used to  evaluate AMR metrics on their sensitivity at detecting structural changes between pairs of AMRs.","description_withheld":null,"homepage":"https://github.com/osome-iu/Rematch-RARE","introduced_date":"2024-04-02","introduced_date_note":null,"introduced_by":{"paper":"/paper/rematch-robust-and-efficient-matching-of","title":"Rematch: Robust and Efficient Matching of Local Knowledge Graphs to Improve Structural and Semantic Similarity","first_author":"Zoher Kachwala","url":null},"license":{"name":"BSD","url":"https://catalog.ldc.upenn.edu/license/ldc-non-members-agreement.pdf"},"modalities":[{"name":"Graphs","url":"/datasets/modality/graphs"}],"tasks":[{"name":"Graph Matching","url":"/task/graph-matching","datasets_with_task":"/datasets/task/graph-matching"}],"languages":[{"name":"English","url":"/datasets/language/english"}],"variants":["RARE"],"data_loaders":[{"repo":"https://github.com/osome-iu/Rematch-RARE","url":"https://github.com/osome-iu/Rematch-RARE","frameworks":[]}],"num_papers_in_archive":5,"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28"},"benchmarks":[{"leaderboard":"/sota/graph-matching-on-rare","task":"Graph Matching","dataset_variant":"RARE","rows":5,"metrics":["Spearman Correlation"],"first_row_in_archive_order":{"model":"Smatch","paper":"/paper/smatch-an-evaluation-metric-for-semantic","metrics":{"Spearman Correlation":"96.57"},"code_links":[{"title":"snowblink14/smatch","url":"https://github.com/snowblink14/smatch"}]},"note":"rows are the archive's own order at snapshot; nothing here re-ranks them"}],"papers_with_a_benchmark_row":[{"paper":"/paper/rematch-robust-and-efficient-matching-of","title":"Rematch: Robust and Efficient Matching of Local Knowledge Graphs to Improve Structural and Semantic Similarity","date":"2024-04-02","rows_on_this_dataset":1,"code_links":1,"syntology":null},{"paper":"/paper/weisfeiler-leman-in-the-bamboo-novel-amr","title":"Weisfeiler-Leman in the BAMBOO: Novel AMR Graph Metrics and a Benchmark for AMR Graph Similarity","date":"2021-08-26","rows_on_this_dataset":1,"code_links":0,"syntology":null},{"paper":"/paper/amr-similarity-metrics-from-principles","title":"AMR Similarity Metrics from Principles","date":"2020-01-29","rows_on_this_dataset":1,"code_links":3,"syntology":null},{"paper":"/paper/sembleu-a-robust-metric-for-amr-parsing","title":"SemBleu: A Robust Metric for AMR Parsing Evaluation","date":"2019-05-26","rows_on_this_dataset":1,"code_links":2,"syntology":null},{"paper":"/paper/smatch-an-evaluation-metric-for-semantic","title":"Smatch: an Evaluation Metric for Semantic Feature Structures","date":"2013-08-01","rows_on_this_dataset":1,"code_links":1,"syntology":null}],"syntology_totals":{"read_at":"2026-09-24T18:15:14+00:00","papers_with_samples":0,"samples_harvested":0,"samples_ran":0,"samples_unverified":0,"pointer_only_for_licence":0,"papers_with_no_sample_that_ran":0,"note":"the per-paper counts above, summed; not a rate"},"papers_note":"The archive never published its papers-using-dataset list; these are papers with a leaderboard row on this dataset's benchmarks."}