Papers › Reranking for Neural Semantic Parsing

Reranking for Neural Semantic Parsing

1 Jul 2019ACL 2019 7archive 2025-07-28

Pengcheng Yin, Graham Neubig

Semantic parsing considers the task of transducing natural language (NL) utterances into machine executable meaning representations (MRs). While neural network-based semantic parsers have achieved impressive improvements over previous methods, results are still far from perfect, and cursory manual inspection can easily identify obvious problems such as lack of adequacy or coherence of the generated MRs. This paper presents a simple approach to quickly iterate and improve the performance of an existing neural semantic parser by reranking an n-best list of predicted MRs, using features that are designed to fix observed problems with baseline models. We implement our reranker in a competitive neural semantic parser and test on four semantic parsing (GEO, ATIS) and Python code generation (Django, CoNaLa) tasks, improving the strong baseline parser by up to 5.7{\%} absolute in BLEU (CoNaLa) and 2.9{\%} in accuracy (Django), outperforming the best published neural parser results on all four datasets.

PaperPDF

Code

No code repository is listed for this paper in the archive or in Syntology's graph.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Code GenerationRerankingSemantic Parsing

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Code Generation CoNaLa Reranker BLEU 30.11 #11 of 14 Archive leaderboard report
Code Generation CoNaLa Reranker Exact Match Accuracy 2.8 #11 of 14 Archive leaderboard report
Code Generation CoNaLa-Ext Reranker BLEU 19.85 #5 of 6 Archive leaderboard report
Code Generation Django Reranker Accuracy 80.2 #4 of 11 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections