{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/autoloss-learning-discrete-schedules-for","title":"AutoLoss: Learning Discrete Schedules for Alternate Optimization","arxiv_id":"1810.02442","date":"2018-10-04","proceeding":null,"authors":["Haowen Xu","Hao Zhang","Zhiting Hu","Xiaodan Liang","Ruslan Salakhutdinov","Eric Xing"],"abstract":"Many machine learning problems involve iteratively and alternately optimizing\ndifferent task objectives with respect to different sets of parameters.\nAppropriately scheduling the optimization of a task objective or a set of\nparameters is usually crucial to the quality of convergence. In this paper, we\npresent AutoLoss, a meta-learning framework that automatically learns and\ndetermines the optimization schedule. AutoLoss provides a generic way to\nrepresent and learn the discrete optimization schedule from metadata, allows\nfor a dynamic and data-driven schedule in ML problems that involve alternating\nupdates of different parameters or from different loss objectives. We apply\nAutoLoss on four ML tasks: d-ary quadratic regression, classification using a\nmulti-layer perceptron (MLP), image generation using GANs, and multi-task\nneural machine translation (NMT). We show that the AutoLoss controller is able\nto capture the distribution of better optimization schedules that result in\nhigher quality of convergence on all four tasks. The trained AutoLoss\ncontroller is generalizable -- it can guide and improve the learning of a new\ntask model with different specifications, or on different datasets.","url_abs":"http://arxiv.org/abs/1810.02442v1","url_pdf":"http://arxiv.org/pdf/1810.02442v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"autoloss-learning-discrete-schedules-for","repo_url":"https://github.com/safpla/AutoLossRelease","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"meta-learning","task_name":"Meta-Learning"},{"task_slug":"nmt","task_name":"NMT"},{"task_slug":"scheduling","task_name":"Scheduling"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1810.02442","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}