{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/effects-of-sampling-skewness-of-the","title":"Effects of sampling skewness of the importance-weighted risk estimator on model selection","arxiv_id":"1804.07344","date":"2018-04-19","proceeding":null,"authors":["Wouter M. Kouw","Marco Loog"],"abstract":"Importance-weighting is a popular and well-researched technique for dealing\nwith sample selection bias and covariate shift. It has desirable\ncharacteristics such as unbiasedness, consistency and low computational\ncomplexity. However, weighting can have a detrimental effect on an estimator as\nwell. In this work, we empirically show that the sampling distribution of an\nimportance-weighted estimator can be skewed. For sample selection bias\nsettings, and for small sample sizes, the importance-weighted risk estimator\nproduces overestimates for datasets in the body of the sampling distribution,\ni.e. the majority of cases, and large underestimates for data sets in the tail\nof the sampling distribution. These over- and underestimates of the risk lead\nto suboptimal regularization parameters when used for importance-weighted\nvalidation.","url_abs":"http://arxiv.org/abs/1804.07344v1","url_pdf":"http://arxiv.org/pdf/1804.07344v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"effects-of-sampling-skewness-of-the","repo_url":"https://github.com/wmkouw/covshift-skewness","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"model-selection","task_name":"Model Selection"},{"task_slug":"selection-bias","task_name":"Selection bias"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}