{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-tunable-loss-function-for-classification","title":"A Tunable Loss Function for Robust Classification: Calibration, Landscape, and Generalization","arxiv_id":"1906.02314","date":"2019-06-05","proceeding":null,"authors":["Tyler Sypherd","Mario Diaz","John Kevin Cava","Gautam Dasarathy","Peter Kairouz","Lalitha Sankar"],"abstract":"We introduce a tunable loss function called $\\alpha$-loss, parameterized by $\\alpha \\in (0,\\infty]$, which interpolates between the exponential loss ($\\alpha = 1/2$), the log-loss ($\\alpha = 1$), and the 0-1 loss ($\\alpha = \\infty$), for the machine learning setting of classification. Theoretically, we illustrate a fundamental connection between $\\alpha$-loss and Arimoto conditional entropy, verify the classification-calibration of $\\alpha$-loss in order to demonstrate asymptotic optimality via Rademacher complexity generalization techniques, and build-upon a notion called strictly local quasi-convexity in order to quantitatively characterize the optimization landscape of $\\alpha$-loss. Practically, we perform class imbalance, robustness, and classification experiments on benchmark image datasets using convolutional-neural-networks. Our main practical conclusion is that certain tasks may benefit from tuning $\\alpha$-loss away from log-loss ($\\alpha = 1$), and to this end we provide simple heuristics for the practitioner. In particular, navigating the $\\alpha$ hyperparameter can readily provide superior model robustness to label flips ($\\alpha > 1$) and sensitivity to imbalanced classes ($\\alpha < 1$).","url_abs":"https://arxiv.org/abs/1906.02314v6","url_pdf":"https://arxiv.org/pdf/1906.02314v6.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-tunable-loss-function-for-classification","repo_url":"https://github.com/sankarlab/alphaloss","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"robust-classification","task_name":"Robust classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1906.02314","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}