{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/what-do-different-evaluation-metrics-tell-us","title":"What do different evaluation metrics tell us about saliency models?","arxiv_id":"1604.03605","date":"2016-04-12","proceeding":null,"authors":["Zoya Bylinskii","Tilke Judd","Aude Oliva","Antonio Torralba","Frédo Durand"],"abstract":"How best to evaluate a saliency model's ability to predict where humans look\nin images is an open research question. The choice of evaluation metric depends\non how saliency is defined and how the ground truth is represented. Metrics\ndiffer in how they rank saliency models, and this results from how false\npositives and false negatives are treated, whether viewing biases are accounted\nfor, whether spatial deviations are factored in, and how the saliency maps are\npre-processed. In this paper, we provide an analysis of 8 different evaluation\nmetrics and their properties. With the help of systematic experiments and\nvisualizations of metric computations, we add interpretability to saliency\nscores and more transparency to the evaluation of saliency models. Building off\nthe differences in metric properties and behaviors, we make recommendations for\nmetric selections under specific assumptions and for specific applications.","url_abs":"http://arxiv.org/abs/1604.03605v2","url_pdf":"http://arxiv.org/pdf/1604.03605v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"what-do-different-evaluation-metrics-tell-us","repo_url":"https://github.com/cvzoya/saliency","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}}],"tasks":[],"methods":[{"method_slug":"interpretability","method_name":"Interpretability"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=1604.03605","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}