{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/generating-the-support-with-extreme-value","title":"Generating the support with extreme value losses","arxiv_id":"1902.02940","date":"2019-02-08","proceeding":null,"authors":["Nicholas Guttenberg"],"abstract":"When optimizing against the mean loss over a distribution of predictions in\nthe context of a regression task, then even if there is a distribution of\ntargets the optimal prediction distribution is always a delta function at a\nsingle value. Methods of constructing generative models need to overcome this\ntendency. We consider a simple method of summarizing the prediction error, such\nthat the optimal strategy corresponds to outputting a distribution of\npredictions with a support that matches the support of the distribution of\ntargets --- optimizing against the minimal value of the loss given a set of\nsamples from the prediction distribution, rather than the mean. We show that\nmodels trained against this loss learn to capture the support of the target\ndistribution and, when combined with an auxiliary classifier-like prediction\ntask, can be projected via rejection sampling to reproduce the full\ndistribution of targets. The resulting method works well compared to other\ngenerative modeling approaches particularly in low dimensional spaces with\nhighly non-trivial distributions, due to mode collapse solutions being globally\nsuboptimal with respect to the extreme value loss. However, the method is less\nsuited to high-dimensional spaces such as images due to the scaling of the\nnumber of samples needed in order to accurately estimate the extreme value loss\nwhen the dimension of the data manifold becomes large.","url_abs":"http://arxiv.org/abs/1902.02940v1","url_pdf":"http://arxiv.org/pdf/1902.02940v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"generating-the-support-with-extreme-value","repo_url":"https://github.com/ngutten/ExtremeValueLoss","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"prediction","task_name":"Prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}