{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mapeval-towards-unified-robust-and-efficient","title":"MapEval: Towards Unified, Robust and Efficient SLAM Map Evaluation Framework","arxiv_id":"2411.17928","date":"2024-11-26","proceeding":null,"authors":["Xiangcheng Hu","Jin Wu","Mingkai Jia","Hongyu Yan","Yi Jiang","Binqian Jiang","Wei Zhang","Wei He","Ping Tan"],"abstract":"Evaluating massive-scale point cloud maps in Simultaneous Localization and Mapping (SLAM) remains challenging, primarily due to the absence of unified, robust and efficient evaluation frameworks. We present MapEval, an open-source framework for comprehensive quality assessment of point cloud maps, specifically addressing SLAM scenarios where ground truth map is inherently sparse compared to the mapped environment. Through systematic analysis of existing evaluation metrics in SLAM applications, we identify their fundamental limitations and establish clear guidelines for consistent map quality assessment. Building upon these insights, we propose a novel Gaussian-approximated Wasserstein distance in voxelized space, enabling two complementary metrics under the same error standard: Voxelized Average Wasserstein Distance (AWD) for global geometric accuracy and Spatial Consistency Score (SCS) for local consistency evaluation. This theoretical foundation leads to significant improvements in both robustness against noise and computational efficiency compared to conventional metrics. Extensive experiments on both simulated and real-world datasets demonstrate that MapEval achieves at least \\SI{100}{}-\\SI{500}{} times faster while maintaining evaluation integrity. The MapEval library\\footnote{\\texttt{https://github.com/JokerJohn/Cloud\\_Map\\_Evaluation}} will be publicly available to promote standardized map evaluation practices in the robotics community.","url_abs":"https://arxiv.org/abs/2411.17928v2","url_pdf":"https://arxiv.org/pdf/2411.17928v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"mapeval-towards-unified-robust-and-efficient","repo_url":"https://github.com/jokerjohn/cloud_map_evaluation","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"mapeval-towards-unified-robust-and-efficient","repo_url":"https://github.com/jokerjohn/paloc","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}