{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/rethinking-the-generalization-of-drug-target","title":"Rethinking the generalization of drug target affinity prediction algorithms via similarity aware evaluation","arxiv_id":"2504.09481","date":"2025-04-13","proceeding":null,"authors":["Chenbin Zhang","Zhiqiang Hu","Chuchu Jiang","Wen Chen","Jie Xu","Shaoting Zhang"],"abstract":"Drug-target binding affinity prediction is a fundamental task for drug discovery. It has been extensively explored in literature and promising results are reported. However, in this paper, we demonstrate that the results may be misleading and cannot be well generalized to real practice. The core observation is that the canonical randomized split of a test set in conventional evaluation leaves the test set dominated by samples with high similarity to the training set. The performance of models is severely degraded on samples with lower similarity to the training set but the drawback is highly overlooked in current evaluation. As a result, the performance can hardly be trusted when the model meets low-similarity samples in real practice. To address this problem, we propose a framework of similarity aware evaluation in which a novel split methodology is proposed to adapt to any desired distribution. This is achieved by a formulation of optimization problems which are approximately and efficiently solved by gradient descent. We perform extensive experiments across five representative methods in four datasets for two typical target evaluations and compare them with various counterpart methods. Results demonstrate that the proposed split methodology can significantly better fit desired distributions and guide the development of models. Code is released at https://github.com/Amshoreline/SAE/tree/main.","url_abs":"https://arxiv.org/abs/2504.09481v1","url_pdf":"https://arxiv.org/pdf/2504.09481v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"rethinking-the-generalization-of-drug-target","repo_url":"https://github.com/amshoreline/sae","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"drug-discovery","task_name":"Drug Discovery"}],"methods":[{"method_slug":"aware","method_name":"AWARE"},{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2504.09481","atlas_url":"https://app.syntology.ai/?focus=2504.09481","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2504.09481"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/Amshoreline/SAE","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/amshoreline/sae","reach":{"status":"ok"}}],"summary":{"ran_fixture":1,"ran_draft_wrong":2},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"bad64604d6d59390","entry":"init_custom","repo":"amshoreline/sae","repo_kind":"official","path":"SAE_split.py","file_url":"https://github.com/amshoreline/sae/blob/HEAD/SAE_split.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bad64604d6d59390"}},{"code_sha256_prefix":"e24607819a71a312","entry":"init_normal","repo":"amshoreline/sae","repo_kind":"official","path":"SAE_split.py","file_url":"https://github.com/amshoreline/sae/blob/HEAD/SAE_split.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e24607819a71a312"}},{"code_sha256_prefix":"6aac5b218f505d76","entry":"init_uniform","repo":"amshoreline/sae","repo_kind":"official","path":"SAE_split.py","file_url":"https://github.com/amshoreline/sae/blob/HEAD/SAE_split.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6aac5b218f505d76"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}