{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/variance-based-regularization-with-convex","title":"Variance-based regularization with convex objectives","arxiv_id":"1610.02581","date":"2016-10-08","proceeding":"NeurIPS 2017 12","authors":["John Duchi","Hongseok Namkoong"],"abstract":"We develop an approach to risk minimization and stochastic optimization that\nprovides a convex surrogate for variance, allowing near-optimal and\ncomputationally efficient trading between approximation and estimation error.\nOur approach builds off of techniques for distributionally robust optimization\nand Owen's empirical likelihood, and we provide a number of finite-sample and\nasymptotic results characterizing the theoretical performance of the estimator.\nIn particular, we show that our procedure comes with certificates of\noptimality, achieving (in some scenarios) faster rates of convergence than\nempirical risk minimization by virtue of automatically balancing bias and\nvariance. We give corroborating empirical evidence showing that in practice,\nthe estimator indeed trades between variance and absolute performance on a\ntraining sample, improving out-of-sample (test) performance over standard\nempirical risk minimization for a number of classification problems.","url_abs":"http://arxiv.org/abs/1610.02581v3","url_pdf":"http://arxiv.org/pdf/1610.02581v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"variance-based-regularization-with-convex","repo_url":"https://github.com/hsnamkoong/robustopt","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"stochastic-optimization","task_name":"Stochastic Optimization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1610.02581","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}