{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/uncertainty-sampling-is-preconditioned","title":"Uncertainty Sampling is Preconditioned Stochastic Gradient Descent on Zero-One Loss","arxiv_id":"1812.01815","date":"2018-12-05","proceeding":"NeurIPS 2018 12","authors":["Stephen Mussmann","Percy Liang"],"abstract":"Uncertainty sampling, a popular active learning algorithm, is used to reduce\nthe amount of data required to learn a classifier, but it has been observed in\npractice to converge to different parameters depending on the initialization\nand sometimes to even better parameters than standard training on all the data.\nIn this work, we give a theoretical explanation of this phenomenon, showing\nthat uncertainty sampling on a convex loss can be interpreted as performing a\npreconditioned stochastic gradient step on a smoothed version of the population\nzero-one loss that converges to the population zero-one loss. Furthermore,\nuncertainty sampling moves in a descent direction and converges to stationary\npoints of the smoothed population zero-one loss. Experiments on synthetic and\nreal datasets support this connection.","url_abs":"http://arxiv.org/abs/1812.01815v1","url_pdf":"http://arxiv.org/pdf/1812.01815v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"uncertainty-sampling-is-preconditioned","repo_url":"https://worksheets.codalab.org/worksheets/0xf8dfe5bcc1dc408fb54b3cc15a5abce8","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"active-learning","task_name":"Active Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1812.01815","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}