Datasets › RARE
RARE (Randomized AMRs with Rewired Edges)
RARE consists of English AMR pairs with similarity scores that reflect the structural differences between them.
Given that AMRs are graphical representations of text, an AMR similarity metric should be sensitive to structural variations between AMRs, even if its labels remain unchanged. Since there is no established evaluation of AMR metrics on structural similarity, we have developed a new benchmark dataset called Randomized AMRs with Rewired Edges (RARE). RARE consists of English AMR pairs with similarity scores that reflect the structural differences between them.
It can be used to evaluate AMR metrics on their sensitivity at detecting structural changes between pairs of AMRs.
Benchmarks archive 2025-07-28
All 1 leaderboard whose dataset resolves to this page shown (sort by any header). "First row" is the archive's own first row at snapshot, in the archive's row order; nothing here re-ranks and metric direction is not asserted.
| First row (archive order) | Paper | Code | ||||
|---|---|---|---|---|---|---|
| Graph Matching | RARE | Smatch Spearman Correlation 96.57 | Smatch: an Evaluation Metric for Semantic Feature Structures | snowblink14/smatch | 5 | Compare |
Papers archive 2025-07-28
5 shown of 5 papers with a leaderboard row on this dataset's benchmarks, newest first. The archive's own "papers using this dataset" list was never published, so this is the benchmark-backed subset; the archive's count for this dataset is 5. The Syntology column is from Syntology's graph (read 2026-09-24), stated per sample; it is not part of any archive number.
| Date | Samples run Syntology | |||
|---|---|---|---|---|
| Rematch: Robust and Efficient Matching of Local Knowledge Graphs to Improve Structural and Semantic Similarity | 1 | 1 | 2 Apr 2024 | not harvested |
| Weisfeiler-Leman in the BAMBOO: Novel AMR Graph Metrics and a Benchmark for AMR Graph Similarity | 0 | 1 | 26 Aug 2021 | not harvested |
| AMR Similarity Metrics from Principles | 3 | 1 | 29 Jan 2020 | not harvested |
| SemBleu: A Robust Metric for AMR Parsing Evaluation | 2 | 1 | 26 May 2019 | not harvested |
| Smatch: an Evaluation Metric for Semantic Feature Structures | 1 | 1 | 1 Aug 2013 | not harvested |
Dataset loaders archive 2025-07-28
1 loader as listed in the archive; links are outbound and not re-checked here.
Tasks archive 2025-07-28
License archive 2025-07-28
Modalities archive 2025-07-28
Languages archive 2025-07-28
Variants archive 2025-07-28
- RARE
1 variant name, as the archive lists them.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections