{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/hit-and-lead-discovery-with-explorative-rl","title":"Hit and Lead Discovery with Explorative RL and Fragment-based Molecule Generation","arxiv_id":"2110.01219","date":"2021-10-04","proceeding":"NeurIPS 2021 12","authors":["Soojung Yang","Doyeong Hwang","Seul Lee","Seongok Ryu","Sung Ju Hwang"],"abstract":"Recently, utilizing reinforcement learning (RL) to generate molecules with desired properties has been highlighted as a promising strategy for drug design. A molecular docking program - a physical simulation that estimates protein-small molecule binding affinity - can be an ideal reward scoring function for RL, as it is a straightforward proxy of the therapeutic potential. Still, two imminent challenges exist for this task. First, the models often fail to generate chemically realistic and pharmacochemically acceptable molecules. Second, the docking score optimization is a difficult exploration problem that involves many local optima and less smooth surfaces with respect to molecular structure. To tackle these challenges, we propose a novel RL framework that generates pharmacochemically acceptable molecules with large docking scores. Our method - Fragment-based generative RL with Explorative Experience replay for Drug design (FREED) - constrains the generated molecules to a realistic and qualified chemical space and effectively explores the space to find drugs by coupling our fragment-based generation method and a novel error-prioritized experience replay (PER). We also show that our model performs well on both de novo and scaffold-based schemes. Our model produces molecules of higher quality compared to existing methods while achieving state-of-the-art performance on two of three targets in terms of the docking scores of the generated molecules. We further show with ablation studies that our method, predictive error-PER (FREED(PE)), significantly improves the model performance.","url_abs":"https://arxiv.org/abs/2110.01219v3","url_pdf":"https://arxiv.org/pdf/2110.01219v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"drug-design","task_name":"Drug Design"},{"task_slug":"molecular-docking","task_name":"Molecular Docking"},{"task_slug":"reinforcement-learning-1","task_name":"Reinforcement Learning (RL)"}],"methods":[{"method_slug":"experience-replay","method_name":"Experience Replay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2110.01219","atlas_url":"https://app.syntology.ai/?focus=2110.01219","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2110.01219"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/AITRICS/FREED","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":1,"ran":3,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"57542f87e248db45","entry":"combined_shape","repo":"AITRICS/FREED","repo_kind":"found_in_text","path":"core_motif.py","file_url":"https://github.com/AITRICS/FREED/blob/HEAD/core_motif.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"57542f87e248db45"}},{"code_sha256_prefix":"4cca5e460067b900","entry":"count_vars","repo":"AITRICS/FREED","repo_kind":"found_in_text","path":"core_motif.py","file_url":"https://github.com/AITRICS/FREED/blob/HEAD/core_motif.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4cca5e460067b900"}},{"code_sha256_prefix":"20c913cffd65870b","entry":"gpu_setup","repo":"AITRICS/FREED","repo_kind":"found_in_text","path":"run_rl.py","file_url":"https://github.com/AITRICS/FREED/blob/HEAD/run_rl.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"20c913cffd65870b"}},{"code_sha256_prefix":"37b5e57246562ac1","entry":"reduce_mean","repo":"AITRICS/FREED","repo_kind":"found_in_text","path":"core_motif.py","file_url":"https://github.com/AITRICS/FREED/blob/HEAD/core_motif.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"37b5e57246562ac1"}},{"code_sha256_prefix":"a3a53aca2a2eedae","entry":"get_att_points","repo":"AITRICS/FREED","repo_kind":"found_in_text","path":"sac_motif_freed_bu.py","file_url":"https://github.com/AITRICS/FREED/blob/HEAD/sac_motif_freed_bu.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a3a53aca2a2eedae"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}