{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/bayesian-algorithm-execution-estimating","title":"Bayesian Algorithm Execution: Estimating Computable Properties of Black-box Functions Using Mutual Information","arxiv_id":"2104.09460","date":"2021-04-19","proceeding":null,"authors":["Willie Neiswanger","Ke Alexander Wang","Stefano Ermon"],"abstract":"In many real-world problems, we want to infer some property of an expensive black-box function $f$, given a budget of $T$ function evaluations. One example is budget constrained global optimization of $f$, for which Bayesian optimization is a popular method. Other properties of interest include local optima, level sets, integrals, or graph-structured information induced by $f$. Often, we can find an algorithm $\\mathcal{A}$ to compute the desired property, but it may require far more than $T$ queries to execute. Given such an $\\mathcal{A}$, and a prior distribution over $f$, we refer to the problem of inferring the output of $\\mathcal{A}$ using $T$ evaluations as Bayesian Algorithm Execution (BAX). To tackle this problem, we present a procedure, InfoBAX, that sequentially chooses queries that maximize mutual information with respect to the algorithm's output. Applying this to Dijkstra's algorithm, for instance, we infer shortest paths in synthetic and real-world graphs with black-box edge costs. Using evolution strategies, we yield variants of Bayesian optimization that target local, rather than global, optima. On these problems, InfoBAX uses up to 500 times fewer queries to $f$ than required by the original algorithm. Our method is closely connected to other Bayesian optimal experimental design procedures such as entropy search methods and optimal sensor placement using Gaussian processes.","url_abs":"https://arxiv.org/abs/2104.09460v2","url_pdf":"https://arxiv.org/pdf/2104.09460v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"bayesian-algorithm-execution-estimating","repo_url":"https://github.com/willieneis/bayesian-algorithm-execution","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"bayesian-optimization","task_name":"Bayesian Optimization"},{"task_slug":"experimental-design","task_name":"Experimental Design"},{"task_slug":"gaussian-processes","task_name":"Gaussian Processes"},{"task_slug":"global-optimization","task_name":"global-optimization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2104.09460","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2104.09460"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/j-wilson/GPflowSampling","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/willieneis/bayesian-algorithm-execution","reach":null}],"summary":{"ran_violates":3,"ran":5,"ran_draft_wrong":2,"unverified":2},"by_repo_kind":{"official":{"samples":12,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":12,"samples":[{"code_sha256_prefix":"1fee3d4f7884fa26","entry":"inv_softplus","repo":"willieneis/bayesian-algorithm-execution","repo_kind":"official","path":"examples/dijkstra/bax_grid10_viz_simple_demo.py","file_url":"https://github.com/willieneis/bayesian-algorithm-execution/blob/HEAD/examples/dijkstra/bax_grid10_viz_simple_demo.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":2,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1fee3d4f7884fa26"}},{"code_sha256_prefix":"51c70e357c70534a","entry":"AcqFunction","repo":"willieneis/bayesian-algorithm-execution","repo_kind":"official","path":"bax/acq/acquisition.py","file_url":"https://github.com/willieneis/bayesian-algorithm-execution/blob/HEAD/bax/acq/acquisition.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"51c70e357c70534a"}},{"code_sha256_prefix":"0f18df3e0a0098f2","entry":"AlgorithmSet","repo":"willieneis/bayesian-algorithm-execution","repo_kind":"official","path":"bax/acq/acquisition.py","file_url":"https://github.com/willieneis/bayesian-algorithm-execution/blob/HEAD/bax/acq/acquisition.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0f18df3e0a0098f2"}},{"code_sha256_prefix":"2692eaa78d8ab634","entry":"Base","repo":"willieneis/bayesian-algorithm-execution","repo_kind":"official","path":"bax/acq/acquisition.py","file_url":"https://github.com/willieneis/bayesian-algorithm-execution/blob/HEAD/bax/acq/acquisition.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2692eaa78d8ab634"}},{"code_sha256_prefix":"89fb3a61f1802ac3","entry":"FunctionSample","repo":"willieneis/bayesian-algorithm-execution","repo_kind":"official","path":"bax/acq/acquisition.py","file_url":"https://github.com/willieneis/bayesian-algorithm-execution/blob/HEAD/bax/acq/acquisition.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"89fb3a61f1802ac3"}},{"code_sha256_prefix":"a8822bf268bce5b0","entry":"Timer","repo":"willieneis/bayesian-algorithm-execution","repo_kind":"official","path":"bax/acq/acquisition.py","file_url":"https://github.com/willieneis/bayesian-algorithm-execution/blob/HEAD/bax/acq/acquisition.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a8822bf268bce5b0"}},{"code_sha256_prefix":"8a3545000b23b4c3","entry":"dict_to_namespace","repo":"willieneis/bayesian-algorithm-execution","repo_kind":"official","path":"bax/acq/acquisition.py","file_url":"https://github.com/willieneis/bayesian-algorithm-execution/blob/HEAD/bax/acq/acquisition.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8a3545000b23b4c3"}},{"code_sha256_prefix":"ffb638b578392224","entry":"rosenbrock","repo":"willieneis/bayesian-algorithm-execution","repo_kind":"official","path":"examples/dijkstra/bax_grid10_viz_simple_demo.py","file_url":"https://github.com/willieneis/bayesian-algorithm-execution/blob/HEAD/examples/dijkstra/bax_grid10_viz_simple_demo.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ffb638b578392224"}},{"code_sha256_prefix":"c65d09efc79848d2","entry":"run_algo_on_mean_f","repo":"willieneis/bayesian-algorithm-execution","repo_kind":"official","path":"examples/branin/bax_viz2d_simple_demo.py","file_url":"https://github.com/willieneis/bayesian-algorithm-execution/blob/HEAD/examples/branin/bax_viz2d_simple_demo.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c65d09efc79848d2"}},{"code_sha256_prefix":"fa8d8e533dc229a6","entry":"true_f","repo":"willieneis/bayesian-algorithm-execution","repo_kind":"official","path":"examples/dijkstra/bax_grid10_viz_simple_demo.py","file_url":"https://github.com/willieneis/bayesian-algorithm-execution/blob/HEAD/examples/dijkstra/bax_grid10_viz_simple_demo.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fa8d8e533dc229a6"}},{"code_sha256_prefix":"a2dc30ca4df3d5df","entry":"AlgoAcqFunction","repo":"willieneis/bayesian-algorithm-execution","repo_kind":"official","path":"bax/acq/acquisition.py","file_url":"https://github.com/willieneis/bayesian-algorithm-execution/blob/HEAD/bax/acq/acquisition.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a2dc30ca4df3d5df"}},{"code_sha256_prefix":"40d6cc52dd9fd829","entry":"BaxAcqFunction","repo":"willieneis/bayesian-algorithm-execution","repo_kind":"official","path":"bax/acq/acquisition.py","file_url":"https://github.com/willieneis/bayesian-algorithm-execution/blob/HEAD/bax/acq/acquisition.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"40d6cc52dd9fd829"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}