{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/2402-16445","title":"ProLLaMA: A Protein Language Model for Multi-Task Protein Language Processing","arxiv_id":"2402.16445","date":"2024-02-26","proceeding":null,"authors":["Liuzhenghao Lv","Zongying Lin","Hao Li","Yuyang Liu","Jiaxi Cui","Calvin Yu-Chian Chen","Li Yuan","Yonghong Tian"],"abstract":"Recent advances in Protein Language Models (PLMs) have transformed protein engineering, yet unlike their counterparts in Natural Language Processing (NLP), current PLMs exhibit a fundamental limitation: they excel in either Protein Language Understanding (PLU) or Protein Language Generation (PLG), but rarely both. This fragmentation hinders progress in protein engineering. To bridge this gap, we introduce ProLLaMA, a multitask protein language model enhanced by the Evolutionary Protein Generation Framework (EPGF). We construct a comprehensive instruction dataset containing approximately 13 million samples with over 11,000 superfamily annotations to facilitate better modeling of sequence-function landscapes. We leverage a two-stage training approach to develop ProLLaMA, a multitask LLM with protein domain expertise. Our EPGF addresses the mismatch between statistic language modeling and biological constraints through three innovations: a multi-dimensional interpretable scorer, hierarchical efficient decoding, and a probabilistic-biophysical joint selection mechanism. Extensive experiments demonstrate that ProLLaMA excels in both unconditional and controllable protein generation tasks, achieving superior structural quality metrics compared to existing PLMs. Additionally, ProLLaMA demonstrates strong understanding capabilities with a 67.1% exact match rate in superfamily prediction. EPGF significantly enhances the biological viability of generated sequences, as evidenced by improved biophysical scores (+4.3%) and structural metrics (+14.5%). The project is available at https://github.com/PKU-YuanGroup/ProLLaMA.","url_abs":"https://arxiv.org/abs/2402.16445v2","url_pdf":"https://arxiv.org/pdf/2402.16445v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"2402-16445","repo_url":"https://github.com/lyu6poshao/prollama","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"2402-16445","repo_url":"https://github.com/pku-yuangroup/prollama","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"2402-16445","repo_url":"https://github.com/howardli1984/ecdformer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"2402-16445","repo_url":"https://github.com/linzy19/taxdiff","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.16445","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.16445"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pku-yuangroup/prollama","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/howardli1984/ecdformer","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lyu6poshao/prollama","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/linzy19/taxdiff","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_fixture":3,"ran_draft_wrong":5,"ran_honours":4,"unverified":5},"by_repo_kind":{"listed":{"samples":14,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"8a38d7c525fad8f1","entry":"accuracy","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"8a38d7c525fad8f1"}},{"code_sha256_prefix":"d6a68e210556f857","entry":"approx_standard_normal_cdf","repo":"linzy19/taxdiff","repo_kind":"listed","path":"diffusion/diffusion_utils.py","file_url":"https://github.com/linzy19/taxdiff/blob/HEAD/diffusion/diffusion_utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d6a68e210556f857"}},{"code_sha256_prefix":"6b376a0abe58a427","entry":"compute_metrics","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"6b376a0abe58a427"}},{"code_sha256_prefix":"ab1c9568b4e13899","entry":"continuous_gaussian_log_likelihood","repo":"linzy19/taxdiff","repo_kind":"listed","path":"diffusion/diffusion_utils.py","file_url":"https://github.com/linzy19/taxdiff/blob/HEAD/diffusion/diffusion_utils.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ab1c9568b4e13899"}},{"code_sha256_prefix":"c92c27c924b517e8","entry":"get_2d_sincos_pos_embed","repo":"linzy19/taxdiff","repo_kind":"listed","path":"models.py","file_url":"https://github.com/linzy19/taxdiff/blob/HEAD/models.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c92c27c924b517e8"}},{"code_sha256_prefix":"3e0fa4efc22272d4","entry":"get_beta_schedule","repo":"linzy19/taxdiff","repo_kind":"listed","path":"diffusion/gaussian_diffusion.py","file_url":"https://github.com/linzy19/taxdiff/blob/HEAD/diffusion/gaussian_diffusion.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3e0fa4efc22272d4"}},{"code_sha256_prefix":"36e30c7fb679ec78","entry":"get_named_beta_schedule","repo":"linzy19/taxdiff","repo_kind":"listed","path":"diffusion/gaussian_diffusion.py","file_url":"https://github.com/linzy19/taxdiff/blob/HEAD/diffusion/gaussian_diffusion.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"36e30c7fb679ec78"}},{"code_sha256_prefix":"f6d7c009a8efb8b7","entry":"mean_flat","repo":"linzy19/taxdiff","repo_kind":"listed","path":"diffusion/gaussian_diffusion.py","file_url":"https://github.com/linzy19/taxdiff/blob/HEAD/diffusion/gaussian_diffusion.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f6d7c009a8efb8b7"}},{"code_sha256_prefix":"03310bba324ae4fb","entry":"modulate","repo":"linzy19/taxdiff","repo_kind":"listed","path":"models.py","file_url":"https://github.com/linzy19/taxdiff/blob/HEAD/models.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"03310bba324ae4fb"}},{"code_sha256_prefix":"8afbfc42c6ea0448","entry":"normal_kl","repo":"linzy19/taxdiff","repo_kind":"listed","path":"diffusion/diffusion_utils.py","file_url":"https://github.com/linzy19/taxdiff/blob/HEAD/diffusion/diffusion_utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8afbfc42c6ea0448"}},{"code_sha256_prefix":"591cff6b55ba6bfb","entry":"prepare_model_for_kbit_training","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"591cff6b55ba6bfb"}},{"code_sha256_prefix":"ea9dbc131adf582e","entry":"space_timesteps","repo":"linzy19/taxdiff","repo_kind":"listed","path":"diffusion/respace.py","file_url":"https://github.com/linzy19/taxdiff/blob/HEAD/diffusion/respace.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ea9dbc131adf582e"}},{"code_sha256_prefix":"e48218d7d73db0b3","entry":"create_named_schedule_sampler","repo":"linzy19/taxdiff","repo_kind":"listed","path":"diffusion/timestep_sampler.py","file_url":"https://github.com/linzy19/taxdiff/blob/HEAD/diffusion/timestep_sampler.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e48218d7d73db0b3"}},{"code_sha256_prefix":"29e52607975cd4b2","entry":"find_model","repo":"linzy19/taxdiff","repo_kind":"listed","path":"sample_protein.py","file_url":"https://github.com/linzy19/taxdiff/blob/HEAD/sample_protein.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"29e52607975cd4b2"}},{"code_sha256_prefix":"665d8a4e8f673a4c","entry":"get_2d_sincos_pos_embed_from_grid","repo":"linzy19/taxdiff","repo_kind":"listed","path":"models.py","file_url":"https://github.com/linzy19/taxdiff/blob/HEAD/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"665d8a4e8f673a4c"}},{"code_sha256_prefix":"a7444354fa921976","entry":"len_select","repo":"linzy19/taxdiff","repo_kind":"listed","path":"data_reader/decoder.py","file_url":"https://github.com/linzy19/taxdiff/blob/HEAD/data_reader/decoder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a7444354fa921976"}},{"code_sha256_prefix":"7278c96e9468a7da","entry":"random_select","repo":"linzy19/taxdiff","repo_kind":"listed","path":"data_reader/decoder.py","file_url":"https://github.com/linzy19/taxdiff/blob/HEAD/data_reader/decoder.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7278c96e9468a7da"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}