{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/reliable-algorithm-selection-for-machine","title":"Reliable algorithm selection for machine learning-guided design","arxiv_id":"2503.20767","date":"2025-03-26","proceeding":null,"authors":["Clara Fannjiang","Ji Won Park"],"abstract":"Algorithms for machine learning-guided design, or design algorithms, use machine learning-based predictions to propose novel objects with desired property values. Given a new design task -- for example, to design novel proteins with high binding affinity to a therapeutic target -- one must choose a design algorithm and specify any hyperparameters and predictive and/or generative models involved. How can these decisions be made such that the resulting designs are successful? This paper proposes a method for design algorithm selection, which aims to select design algorithms that will produce a distribution of design labels satisfying a user-specified success criterion -- for example, that at least ten percent of designs' labels exceed a threshold. It does so by combining designs' predicted property values with held-out labeled data to reliably forecast characteristics of the label distributions produced by different design algorithms, building upon techniques from prediction-powered inference. The method is guaranteed with high probability to return design algorithms that yield successful label distributions (or the null set if none exist), if the density ratios between the design and labeled data distributions are known. We demonstrate the method's effectiveness in simulated protein and RNA design tasks, in settings with either known or estimated density ratios.","url_abs":"https://arxiv.org/abs/2503.20767v1","url_pdf":"https://arxiv.org/pdf/2503.20767v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[],"methods":[{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2503.20767","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.20767"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/clarafy/design-algorithm-selection","reach":null}],"summary":{"ran":1,"ran_honours":2,"unverified":2},"by_repo_kind":{"found_in_text":{"samples":5,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"dad11c93ae86fe61","entry":"VAE","repo":"clarafy/design-algorithm-selection","repo_kind":"found_in_text","path":"designers.py","file_url":"https://github.com/clarafy/design-algorithm-selection/blob/HEAD/designers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dad11c93ae86fe61"}},{"code_sha256_prefix":"bdf609fb7465e208","entry":"str2onehot","repo":"clarafy/design-algorithm-selection","repo_kind":"found_in_text","path":"designers.py","file_url":"https://github.com/clarafy/design-algorithm-selection/blob/HEAD/designers.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bdf609fb7465e208"}},{"code_sha256_prefix":"bf0a3f2dc2500465","entry":"type_check_and_one_hot_encode_sequences","repo":"clarafy/design-algorithm-selection","repo_kind":"found_in_text","path":"designers.py","file_url":"https://github.com/clarafy/design-algorithm-selection/blob/HEAD/designers.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bf0a3f2dc2500465"}},{"code_sha256_prefix":"8d94637211d63564","entry":"CbAS","repo":"clarafy/design-algorithm-selection","repo_kind":"found_in_text","path":"designers.py","file_url":"https://github.com/clarafy/design-algorithm-selection/blob/HEAD/designers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8d94637211d63564"}},{"code_sha256_prefix":"959e51ba209b71da","entry":"Designer","repo":"clarafy/design-algorithm-selection","repo_kind":"found_in_text","path":"designers.py","file_url":"https://github.com/clarafy/design-algorithm-selection/blob/HEAD/designers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"959e51ba209b71da"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}