{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/theoretical-analysis-of-bayesian-optimisation","title":"Theoretical Analysis of Bayesian Optimisation with Unknown Gaussian Process Hyper-Parameters","arxiv_id":"1406.7758","date":"2014-06-30","proceeding":null,"authors":["Ziyu Wang","Nando de Freitas"],"abstract":"Bayesian optimisation has gained great popularity as a tool for optimising\nthe parameters of machine learning algorithms and models. Somewhat ironically,\nsetting up the hyper-parameters of Bayesian optimisation methods is notoriously\nhard. While reasonable practical solutions have been advanced, they can often\nfail to find the best optima. Surprisingly, there is little theoretical\nanalysis of this crucial problem in the literature. To address this, we derive\na cumulative regret bound for Bayesian optimisation with Gaussian processes and\nunknown kernel hyper-parameters in the stochastic setting. The bound, which\napplies to the expected improvement acquisition function and sub-Gaussian\nobservation noise, provides us with guidelines on how to design hyper-parameter\nestimation methods. A simple simulation demonstrates the importance of\nfollowing these guidelines.","url_abs":"http://arxiv.org/abs/1406.7758v1","url_pdf":"http://arxiv.org/pdf/1406.7758v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"theoretical-analysis-of-bayesian-optimisation","repo_url":"https://github.com/weepingwillowben/hyperparam-opt","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"bayesian-optimisation","task_name":"Bayesian Optimisation"},{"task_slug":"gaussian-processes","task_name":"Gaussian Processes"},{"task_slug":"parameter-estimation","task_name":"parameter estimation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1406.7758","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1406.7758"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/weepingwillowben/hyperparam-opt","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":6},"by_repo_kind":{"listed":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f126c536091c80c0","entry":"borehole","repo":"weepingwillowben/hyperparam-opt","repo_kind":"listed","path":"borehole.py","file_url":"https://github.com/weepingwillowben/hyperparam-opt/blob/HEAD/borehole.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f126c536091c80c0"}},{"code_sha256_prefix":"a4ed8c98004117c1","entry":"direct","repo":"weepingwillowben/hyperparam-opt","repo_kind":"listed","path":"mydirect.py","file_url":"https://github.com/weepingwillowben/hyperparam-opt/blob/HEAD/mydirect.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a4ed8c98004117c1"}},{"code_sha256_prefix":"4f54de85b54cc7aa","entry":"init_small_vals","repo":"weepingwillowben/hyperparam-opt","repo_kind":"listed","path":"init_optimizer.py","file_url":"https://github.com/weepingwillowben/hyperparam-opt/blob/HEAD/init_optimizer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4f54de85b54cc7aa"}},{"code_sha256_prefix":"3a1158a3fcab04aa","entry":"low_fidelity_borehole","repo":"weepingwillowben/hyperparam-opt","repo_kind":"listed","path":"borehole.py","file_url":"https://github.com/weepingwillowben/hyperparam-opt/blob/HEAD/borehole.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3a1158a3fcab04aa"}},{"code_sha256_prefix":"0f36d9a43a155f5d","entry":"parse_data","repo":"weepingwillowben/hyperparam-opt","repo_kind":"listed","path":"next_point.py","file_url":"https://github.com/weepingwillowben/hyperparam-opt/blob/HEAD/next_point.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0f36d9a43a155f5d"}},{"code_sha256_prefix":"9f9dfaecff7419d8","entry":"read_json","repo":"weepingwillowben/hyperparam-opt","repo_kind":"listed","path":"add_point.py","file_url":"https://github.com/weepingwillowben/hyperparam-opt/blob/HEAD/add_point.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9f9dfaecff7419d8"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}