{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/novobench-benchmarking-deep-learning-based-de","title":"NovoBench: Benchmarking Deep Learning-based De Novo Peptide Sequencing Methods in Proteomics","arxiv_id":"2406.11906","date":"2024-06-16","proceeding":null,"authors":["Jingbo Zhou","Shaorong Chen","Jun Xia","Sizhe Liu","Tianze Ling","Wenjie Du","Yue Liu","Jianwei Yin","Stan Z. Li"],"abstract":"Tandem mass spectrometry has played a pivotal role in advancing proteomics, enabling the high-throughput analysis of protein composition in biological tissues. Many deep learning methods have been developed for \\emph{de novo} peptide sequencing task, i.e., predicting the peptide sequence for the observed mass spectrum. However, two key challenges seriously hinder the further advancement of this important task. Firstly, since there is no consensus for the evaluation datasets, the empirical results in different research papers are often not comparable, leading to unfair comparison. Secondly, the current methods are usually limited to amino acid-level or peptide-level precision and recall metrics. In this work, we present the first unified benchmark NovoBench for \\emph{de novo} peptide sequencing, which comprises diverse mass spectrum data, integrated models, and comprehensive evaluation metrics. Recent impressive methods, including DeepNovo, PointNovo, Casanovo, InstaNovo, AdaNovo and $\\pi$-HelixNovo are integrated into our framework. In addition to amino acid-level and peptide-level precision and recall, we evaluate the models' performance in terms of identifying post-tranlational modifications (PTMs), efficiency and robustness to peptide length, noise peaks and missing fragment ratio, which are important influencing factors while seldom be considered. Leveraging this benchmark, we conduct a large-scale study of current methods, report many insightful findings that open up new possibilities for future development.","url_abs":"https://arxiv.org/abs/2406.11906v2","url_pdf":"https://arxiv.org/pdf/2406.11906v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"de-novo-peptide-sequencing","task_name":"de novo peptide sequencing"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.11906","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.11906"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/Westlake-OmicsAI/NovoBench","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":3},"by_repo_kind":{"found_in_text":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"046e71c75d9ba9b1","entry":"collate_batch","repo":"Westlake-OmicsAI/NovoBench","repo_kind":"found_in_text","path":"novobench/models/adanovo/adanovo_dataloader.py","file_url":"https://github.com/Westlake-OmicsAI/NovoBench/blob/HEAD/novobench/models/adanovo/adanovo_dataloader.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"046e71c75d9ba9b1"}},{"code_sha256_prefix":"1d4cd40b55e04c9d","entry":"process_spectrum","repo":"Westlake-OmicsAI/NovoBench","repo_kind":"found_in_text","path":"novobench/models/deepnovo/deepnovo_dataloader.py","file_url":"https://github.com/Westlake-OmicsAI/NovoBench/blob/HEAD/novobench/models/deepnovo/deepnovo_dataloader.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1d4cd40b55e04c9d"}},{"code_sha256_prefix":"39e1f2d7cabeb996","entry":"split_peptide","repo":"Westlake-OmicsAI/NovoBench","repo_kind":"found_in_text","path":"novobench/metrics/evaluate.py","file_url":"https://github.com/Westlake-OmicsAI/NovoBench/blob/HEAD/novobench/metrics/evaluate.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"39e1f2d7cabeb996"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}