{"url":"/method/codet5","slug":"codet5","name":"CodeT5","full_name":"CodeT5","full_name_withheld":false,"description_markdown":"**CodeT5** is a [Transformer](https://paperswithcode.com/methods/category/transformers)-based model for code understanding and generation based on the [T5 architecture](https://paperswithcode.com/method/t5). It utilizes an identifier-aware pre-training objective that considers the crucial token type information (identifiers) from code. Specifically, the denoising [Seq2Seq](https://paperswithcode.com/method/seq2seq) objective of T5 is extended with two identifier tagging and prediction tasks to enable the model to better leverage the token type information from programming languages, which are the identifiers assigned by developers. To improve the natural language-programming language alignment, a bimodal dual learning objective is used for a bidirectional conversion between natural language and programming language.","description_state":"present","introduced_year":null,"introduced_by":{"title":"CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation","paper":"/paper/codet5-identifier-aware-unified-pre-trained","first_author":"Yue Wang","n_authors":4,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/codet5-identifier-aware-unified-pre-trained"},"source":{"url":"https://arxiv.org/abs/2109.00859v1","title":"CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Natural Language Processing","area_id":"natural-language-processing","collection":"Transformers","url":"/methods/category/transformers","pwc_aliases":[]},{"area":"Natural Language Processing","area_id":"natural-language-processing","collection":"Autoencoding Transformers","url":"/methods/category/autoencoding-transformers","pwc_aliases":[]},{"area":"Natural Language Processing","area_id":"natural-language-processing","collection":"Code Generation Transformers","url":"/methods/category/code-generation-transformers","pwc_aliases":[]}],"n_papers_tagged":32,"archive_num_papers":32,"papers_newest_first":[{"paper":null,"title":"I Know Which LLM Wrote Your Code Last Summer: LLM generated Code Stylometry for Authorship Attribution","date":"2025-06-18","arxiv_id":"2506.17323","n_code_links":0,"syntology":null},{"paper":null,"title":"A Multi-Dataset Evaluation of Models for Automated Vulnerability Repair","date":"2025-06-05","arxiv_id":"2506.04987","n_code_links":0,"syntology":null},{"paper":"/paper/shioenv-a-cli-behavior-capturing-environment","title":"ShIOEnv: A CLI Behavior-Capturing Environment Enabling Grammar-Guided Command Synthesis for Dataset Curation","date":"2025-05-23","arxiv_id":"2505.18374","n_code_links":1,"syntology":null},{"paper":"/paper/logicase-effective-test-case-generation-from","title":"LogiCase: Effective Test Case Generation from Logical Description in Competitive Programming","date":"2025-05-21","arxiv_id":"2505.15039","n_code_links":0,"syntology":{"ran":11,"of":19,"unverified":8,"pointer_only":19}},{"paper":null,"title":"Large Language Models are Qualified Benchmark Builders: Rebuilding Pre-Training Datasets for Advancing Code Intelligence Tasks","date":"2025-04-28","arxiv_id":"2504.19444","n_code_links":0,"syntology":null},{"paper":null,"title":"Enhancing Code LLM Training with Programmer Attention","date":"2025-03-19","arxiv_id":"2503.14936","n_code_links":0,"syntology":null},{"paper":null,"title":"Robust and Secure Code Watermarking for Large Language Models via ML/Crypto Codesign","date":"2025-02-04","arxiv_id":"2502.02068","n_code_links":0,"syntology":null},{"paper":"/paper/how-to-select-pre-trained-code-models-for","title":"How to Select Pre-Trained Code Models for Reuse? A Learning Perspective","date":"2025-01-07","arxiv_id":"2501.03783","n_code_links":1,"syntology":null},{"paper":null,"title":"Can LLMs Obfuscate Code? A Systematic Analysis of Large Language Models into Assembly Code Obfuscation","date":"2024-12-20","arxiv_id":"2412.16135","n_code_links":0,"syntology":null},{"paper":null,"title":"Generative Fuzzy System for Sequence Generation","date":"2024-11-21","arxiv_id":"2411.13867","n_code_links":0,"syntology":null},{"paper":"/paper/building-a-coding-assistant-via-the-retrieval","title":"Building A Coding Assistant via the Retrieval-Augmented Language Model","date":"2024-10-21","arxiv_id":"2410.16229","n_code_links":1,"syntology":null},{"paper":null,"title":"Detection Made Easy: Potentials of Large Language Models for Solidity Vulnerabilities","date":"2024-09-15","arxiv_id":"2409.10574","n_code_links":0,"syntology":null},{"paper":null,"title":"VulCatch: Enhancing Binary Vulnerability Detection through CodeT5 Decompilation and KAN Advanced Feature Extraction","date":"2024-08-13","arxiv_id":"2408.07181","n_code_links":0,"syntology":null},{"paper":"/paper/empirical-studies-of-parameter-efficient","title":"Empirical Studies of Parameter Efficient Methods for Large Language Models of Code and Knowledge Transfer to R","date":"2024-03-16","arxiv_id":"2405.01553","n_code_links":1,"syntology":null},{"paper":"/paper/ast-t5-structure-aware-pretraining-for-code","title":"AST-T5: Structure-Aware Pretraining for Code Generation and Understanding","date":"2024-01-05","arxiv_id":"2401.03003","n_code_links":1,"syntology":{"ran":5,"of":6,"unverified":1,"pointer_only":6}},{"paper":null,"title":"PerfRL: A Small Language Model Framework for Efficient Code Optimization","date":"2023-12-09","arxiv_id":"2312.05657","n_code_links":0,"syntology":null},{"paper":null,"title":"Converting Epics/Stories into Pseudocode using Transformers","date":"2023-12-08","arxiv_id":"2312.05047","n_code_links":0,"syntology":null},{"paper":null,"title":"Learning Defect Prediction from Unrealistic Data","date":"2023-11-02","arxiv_id":"2311.00931","n_code_links":0,"syntology":null},{"paper":"/paper/data-augmentation-for-code-translation-with","title":"Data Augmentation for Code Translation with Comparable Corpora and Multiple References","date":"2023-11-01","arxiv_id":"2311.00317","n_code_links":1,"syntology":{"ran":6,"of":12,"unverified":6,"pointer_only":12}},{"paper":null,"title":"Program Repair with Minimal Edits Using CodeT5","date":"2023-09-26","arxiv_id":"2309.14760","n_code_links":0,"syntology":null},{"paper":null,"title":"RAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repair","date":"2023-09-12","arxiv_id":"2309.06057","n_code_links":0,"syntology":null},{"paper":null,"title":"Domain Adaptation for Code Model-based Unit Test Case Generation","date":"2023-08-15","arxiv_id":"2308.08033","n_code_links":0,"syntology":null},{"paper":null,"title":"Exploring Continual Learning for Code Generation Models","date":"2023-07-05","arxiv_id":"2307.02435","n_code_links":0,"syntology":null},{"paper":"/paper/attention-compilation-and-solver-based","title":"CoTran: An LLM-based Code Translator using Reinforcement Learning with Feedback from Compiler and Symbolic Execution","date":"2023-06-11","arxiv_id":"2306.06755","n_code_links":1,"syntology":null},{"paper":null,"title":"Coeditor: Leveraging Contextual Changes for Multi-round Code Auto-editing","date":"2023-05-29","arxiv_id":"2305.18584","n_code_links":0,"syntology":null},{"paper":"/paper/how-effective-are-neural-networks-for-fixing","title":"How Effective Are Neural Networks for Fixing Security Vulnerabilities","date":"2023-05-29","arxiv_id":"2305.18607","n_code_links":1,"syntology":{"ran":4,"of":4,"unverified":0,"pointer_only":4}},{"paper":null,"title":"GrACE: Generation using Associated Code Edits","date":"2023-05-23","arxiv_id":"2305.14129","n_code_links":0,"syntology":null},{"paper":null,"title":"A Black-Box Attack on Code Models via Representation Nearest Neighbor Search","date":"2023-05-10","arxiv_id":"2305.05896","n_code_links":0,"syntology":null},{"paper":null,"title":"Enhancing Automated Program Repair through Fine-tuning and Prompt Engineering","date":"2023-04-16","arxiv_id":"2304.07840","n_code_links":0,"syntology":null},{"paper":"/paper/a-comprehensive-survey-of-ai-generated","title":"A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT","date":"2023-03-07","arxiv_id":"2303.04226","n_code_links":1,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/code-generation","name":"Code Generation","papers":8},{"task":"/task/code-summarization-1","name":"Code Summarization","papers":6},{"task":"/task/language-modelling","name":"Language Modelling","papers":5},{"task":"/task/program-repair","name":"Program Repair","papers":5},{"task":"/task/vulnerability-detection","name":"Vulnerability Detection","papers":4},{"task":"/task/code-completion","name":"Code Completion","papers":3},{"task":"/task/code-translation","name":"Code Translation","papers":3},{"task":"/task/decoder","name":"Decoder","papers":3},{"task":"/task/clone-detection","name":"Clone Detection","papers":2},{"task":"/task/language-modeling","name":"Language Modeling","papers":2},{"task":"/task/machine-translation","name":"Machine Translation","papers":2},{"task":"/task/retrieval","name":"Retrieval","papers":2},{"task":"/task/small-language-model","name":"Small Language Model","papers":2},{"task":"/task/translation","name":"Translation","papers":2},{"task":"/task/adversarial-attack","name":"Adversarial Attack","papers":1},{"task":"/task/authorship-attribution","name":"Authorship Attribution","papers":1},{"task":"/task/binary-classification","name":"Binary Classification","papers":1},{"task":"/task/bug-fixing","name":"Bug fixing","papers":1},{"task":"/task/code-search","name":"Code Search","papers":1},{"task":"/task/continual-learning","name":"Continual Learning","papers":1}],"tasks_shown":20,"n_tasks":41,"usage_by_year":[{"year":"2015","papers":1},{"year":"2021","papers":1},{"year":"2023","papers":15},{"year":"2024","papers":7},{"year":"2025","papers":8}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/codet5"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}