{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/does-distributionally-robust-supervised","title":"Does Distributionally Robust Supervised Learning Give Robust Classifiers?","arxiv_id":"1611.02041","date":"2016-11-07","proceeding":"ICML 2018 7","authors":["Weihua Hu","Gang Niu","Issei Sato","Masashi Sugiyama"],"abstract":"Distributionally Robust Supervised Learning (DRSL) is necessary for building\nreliable machine learning systems. When machine learning is deployed in the\nreal world, its performance can be significantly degraded because test data may\nfollow a different distribution from training data. DRSL with f-divergences\nexplicitly considers the worst-case distribution shift by minimizing the\nadversarially reweighted training loss. In this paper, we analyze this DRSL,\nfocusing on the classification scenario. Since the DRSL is explicitly\nformulated for a distribution shift scenario, we naturally expect it to give a\nrobust classifier that can aggressively handle shifted distributions. However,\nsurprisingly, we prove that the DRSL just ends up giving a classifier that\nexactly fits the given training distribution, which is too pessimistic. This\npessimism comes from two sources: the particular losses used in classification\nand the fact that the variety of distributions to which the DRSL tries to be\nrobust is too wide. Motivated by our analysis, we propose simple DRSL that\novercomes this pessimism and empirically demonstrate its effectiveness.","url_abs":"http://arxiv.org/abs/1611.02041v6","url_pdf":"http://arxiv.org/pdf/1611.02041v6.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"machine-learning","task_name":"BIG-bench Machine Learning"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"image-classification","task_name":"Image Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-classification-on-iwildcam2020-wilds","task":"Image Classification","dataset":"iWildCam2020-WILDS","model":"Group DRO","rank_in_archive_order":4,"of":6,"metrics":{"Accuracy (Top-1)":"72.7"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1611.02041","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}