{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bayesian-optimization-with-gradients","title":"Bayesian Optimization with Gradients","arxiv_id":"1703.04389","date":"2017-03-13","proceeding":"NeurIPS 2017 12","authors":["Jian Wu","Matthias Poloczek","Andrew Gordon Wilson","Peter I. Frazier"],"abstract":"Bayesian optimization has been successful at global optimization of\nexpensive-to-evaluate multimodal objective functions. However, unlike most\noptimization methods, Bayesian optimization typically does not use derivative\ninformation. In this paper we show how Bayesian optimization can exploit\nderivative information to decrease the number of objective function evaluations\nrequired for good performance. In particular, we develop a novel Bayesian\noptimization algorithm, the derivative-enabled knowledge-gradient (dKG), for\nwhich we show one-step Bayes-optimality, asymptotic consistency, and greater\none-step value of information than is possible in the derivative-free setting.\nOur procedure accommodates noisy and incomplete derivative information, comes\nin both sequential and batch forms, and can optionally reduce the computational\ncost of inference through automatically selected retention of a single\ndirectional derivative. We also compute the d-KG acquisition function and its\ngradient using a novel fast discretization-free technique. We show d-KG\nprovides state-of-the-art performance compared to a wide range of optimization\nprocedures with and without gradients, on benchmarks including logistic\nregression, deep learning, kernel learning, and k-nearest neighbors.","url_abs":"http://arxiv.org/abs/1703.04389v3","url_pdf":"http://arxiv.org/pdf/1703.04389v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bayesian-optimization-with-gradients","repo_url":"https://github.com/wujian16/Cornell-MOE","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}}],"tasks":[{"task_slug":"bayesian-optimization","task_name":"Bayesian Optimization"},{"task_slug":"global-optimization","task_name":"global-optimization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1703.04389","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}