{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dissecting-the-impact-of-different-loss","title":"Dissecting the impact of different loss functions with gradient surgery","arxiv_id":"2201.11307","date":"2022-01-27","proceeding":null,"authors":["Hong Xuan","Robert Pless"],"abstract":"Pair-wise loss is an approach to metric learning that learns a semantic embedding by optimizing a loss function that encourages images from the same semantic class to be mapped closer than images from different classes. The literature reports a large and growing set of variations of the pair-wise loss strategies. Here we decompose the gradient of these loss functions into components that relate to how they push the relative feature positions of the anchor-positive and anchor-negative pairs. This decomposition allows the unification of a large collection of current pair-wise loss functions. Additionally, explicitly constructing pair-wise gradient updates to separate out these effects gives insights into which have the biggest impact, and leads to a simple algorithm that beats the state of the art for image retrieval on the CAR, CUB and Stanford Online products datasets.","url_abs":"https://arxiv.org/abs/2201.11307v1","url_pdf":"https://arxiv.org/pdf/2201.11307v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"image-retrieval","task_name":"Image Retrieval"},{"task_slug":"metric-learning","task_name":"Metric Learning"},{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/metric-learning-on-cars196","task":"Metric Learning","dataset":"CARS196","model":"Gradient Surgery","rank_in_archive_order":24,"of":36,"metrics":{"R@1":"86.5"},"uses_additional_data":false},{"leaderboard":"/sota/metric-learning-on-cub-200-2011","task":"Metric Learning","dataset":"CUB-200-2011","model":"Gradient Surgery","rank_in_archive_order":23,"of":30,"metrics":{"R@1":"63.8"},"uses_additional_data":false},{"leaderboard":"/sota/metric-learning-on-in-shop-1","task":"Metric Learning","dataset":"In-Shop","model":"Gradient Surgery","rank_in_archive_order":7,"of":15,"metrics":{"R@1":"92.21"},"uses_additional_data":false},{"leaderboard":"/sota/metric-learning-on-stanford-online-products-1","task":"Metric Learning","dataset":"Stanford Online Products","model":"Gradient Surgery","rank_in_archive_order":12,"of":33,"metrics":{"R@1":"82.3"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}