{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/scalable-bayesian-optimization-using-deep","title":"Scalable Bayesian Optimization Using Deep Neural Networks","arxiv_id":"1502.05700","date":"2015-02-19","proceeding":null,"authors":["Jasper Snoek","Oren Rippel","Kevin Swersky","Ryan Kiros","Nadathur Satish","Narayanan Sundaram","Md. Mostofa Ali Patwary","Prabhat","Ryan P. Adams"],"abstract":"Bayesian optimization is an effective methodology for the global optimization\nof functions with expensive evaluations. It relies on querying a distribution\nover functions defined by a relatively cheap surrogate model. An accurate model\nfor this distribution over functions is critical to the effectiveness of the\napproach, and is typically fit using Gaussian processes (GPs). However, since\nGPs scale cubically with the number of observations, it has been challenging to\nhandle objectives whose optimization requires many evaluations, and as such,\nmassively parallelizing the optimization.\n  In this work, we explore the use of neural networks as an alternative to GPs\nto model distributions over functions. We show that performing adaptive basis\nfunction regression with a neural network as the parametric form performs\ncompetitively with state-of-the-art GP-based approaches, but scales linearly\nwith the number of data rather than cubically. This allows us to achieve a\npreviously intractable degree of parallelism, which we apply to large scale\nhyperparameter optimization, rapidly finding competitive models on benchmark\nobject recognition tasks using convolutional networks, and image caption\ngeneration using neural language models.","url_abs":"http://arxiv.org/abs/1502.05700v2","url_pdf":"http://arxiv.org/pdf/1502.05700v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"scalable-bayesian-optimization-using-deep","repo_url":"https://github.com/0h-n0/tfdbonas","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"scalable-bayesian-optimization-using-deep","repo_url":"https://github.com/0h-n0/thdbonas","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"scalable-bayesian-optimization-using-deep","repo_url":"https://github.com/automl/pybnn","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"scalable-bayesian-optimization-using-deep","repo_url":"https://github.com/pipilurj/BONAS","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"bayesian-optimization","task_name":"Bayesian Optimization"},{"task_slug":"caption-generation","task_name":"Caption Generation"},{"task_slug":"gaussian-processes","task_name":"Gaussian Processes"},{"task_slug":"hyperparameter-optimization","task_name":"Hyperparameter Optimization"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"object-recognition","task_name":"Object Recognition"},{"task_slug":"global-optimization","task_name":"global-optimization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-classification-on-cifar-10","task":"Image Classification","dataset":"CIFAR-10","model":"Tuned CNN","rank_in_archive_order":173,"of":265,"metrics":{"Percentage correct":"93.6"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-cifar-100","task":"Image Classification","dataset":"CIFAR-100","model":"Tuned CNN","rank_in_archive_order":166,"of":211,"metrics":{"Percentage correct":"72.6"},"uses_additional_data":true}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1502.05700","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1502.05700"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/automl/pybnn","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/0h-n0/tfdbonas","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pipilurj/BONAS","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/0h-n0/thdbonas","reach":{"status":"ok"}}],"summary":{"ran_honours":1},"by_repo_kind":{"listed":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"fa3bc4ad3de49b5f","entry":"toy_example","repo":"automl/pybnn","repo_kind":"listed","path":"examples/example_lc_extrapolation.py","file_url":"https://github.com/automl/pybnn/blob/HEAD/examples/example_lc_extrapolation.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"BSD-3-Clause","inline_ok":true,"mcp_get_code":{"code_sha256":"fa3bc4ad3de49b5f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}