{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/peer-a-comprehensive-and-multi-task-benchmark","title":"PEER: A Comprehensive and Multi-Task Benchmark for Protein Sequence Understanding","arxiv_id":"2206.02096","date":"2022-06-05","proceeding":null,"authors":["Minghao Xu","Zuobai Zhang","Jiarui Lu","Zhaocheng Zhu","Yangtian Zhang","Chang Ma","Runcheng Liu","Jian Tang"],"abstract":"We are now witnessing significant progress of deep learning methods in a variety of tasks (or datasets) of proteins. However, there is a lack of a standard benchmark to evaluate the performance of different methods, which hinders the progress of deep learning in this field. In this paper, we propose such a benchmark called PEER, a comprehensive and multi-task benchmark for Protein sEquence undERstanding. PEER provides a set of diverse protein understanding tasks including protein function prediction, protein localization prediction, protein structure prediction, protein-protein interaction prediction, and protein-ligand interaction prediction. We evaluate different types of sequence-based methods for each task including traditional feature engineering approaches, different sequence encoding methods as well as large-scale pre-trained protein language models. In addition, we also investigate the performance of these methods under the multi-task learning setting. Experimental results show that large-scale pre-trained protein language models achieve the best performance for most individual tasks, and jointly training multiple tasks further boosts the performance. The datasets and source codes of this benchmark are all available at https://github.com/DeepGraphLearning/PEER_Benchmark","url_abs":"https://arxiv.org/abs/2206.02096v2","url_pdf":"https://arxiv.org/pdf/2206.02096v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"peer-a-comprehensive-and-multi-task-benchmark","repo_url":"https://github.com/deepgraphlearning/peer_benchmark","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"feature-engineering","task_name":"Feature Engineering"},{"task_slug":"multi-task-learning","task_name":"Multi-Task Learning"},{"task_slug":"prediction","task_name":"Prediction"},{"task_slug":"protein-function-prediction","task_name":"Protein Function Prediction"},{"task_slug":"protein-structure-prediction","task_name":"Protein Structure Prediction"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2206.02096","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2206.02096"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/DeepGraphLearning/PEER_Benchmark","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/deepgraphlearning/peer_benchmark","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":2},"by_repo_kind":{"official":{"samples":2,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6cdd2cf240cdfce8","entry":"train_and_validate","repo":"DeepGraphLearning/PEER_Benchmark","repo_kind":"official","path":"script/run_single.py","file_url":"https://github.com/DeepGraphLearning/PEER_Benchmark/blob/HEAD/script/run_single.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6cdd2cf240cdfce8"}},{"code_sha256_prefix":"2cb1b6e4c11814b4","entry":"train_and_validate","repo":"DeepGraphLearning/PEER_Benchmark","repo_kind":"official","path":"script/run_multi.py","file_url":"https://github.com/DeepGraphLearning/PEER_Benchmark/blob/HEAD/script/run_multi.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2cb1b6e4c11814b4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}