Methods › Natural Language Processing › Autoencoding Transformers › T5
T5
Introduced by Colin Raffel et al. in Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
T5, or Text-to-Text Transfer Transformer, is a Transformer based architecture that uses a text-to-text approach. Every task – including translation, question answering, and classification – is cast as feeding the model text as input and training it to generate some target text. This allows for the use of the same model, loss function, hyperparameters, etc. across our diverse set of tasks. The changes compared to BERT include:
- adding a causal decoder to the bidirectional architecture.
- replacing the fill-in-the-blank cloze task with a mix of alternative pre-training tasks.
In Syntology name not resolved (unknown_name). The lookup matched no method, paper or venue by this name; that asserts nothing about whether code exists.
Near matches the lookup lists, each a different name: AST-T5; HTML-T5; Flan-T5; Graphix-T5; T5-CSBoost; Definition Generation via Flan-T5.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 708 · with a code link: 353 · where Syntology ran a sample: 99 (85 with a run with no instrument failure, 14 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (99 of 708 tagged: 85 with a run with no instrument failure, 14 where every run was a failure of Syntology's instrument)
Page 1 of 8: papers 1 to 100 of 708, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
LiLM-RDB-SFC: Lightweight Language Model with Relational Database-Guided DRL for Optimized SFC Provisioning 15 Jul 2025 · 0 repositories · arXiv:2507.10903
-
Chat-Ghosting: A Comparative Study of Methods for Auto-Completion in Dialog Systems 8 Jul 2025 · 0 repositories · arXiv:2507.05940
-
I Know Which LLM Wrote Your Code Last Summer: LLM generated Code Stylometry for Authorship Attribution 18 Jun 2025 · 0 repositories · arXiv:2506.17323
-
Fretting-Transformer: Encoder-Decoder Model for MIDI to Tablature Transcription 17 Jun 2025 · 0 repositories · arXiv:2506.14223
-
A Comprehensive Study of Decoder-Only LLMs for Text-to-Image Generation 9 Jun 2025 · 0 repositories · arXiv:2506.08210
-
A Multi-Dataset Evaluation of Models for Automated Vulnerability Repair 5 Jun 2025 · 0 repositories · arXiv:2506.04987
-
Decom-Renorm-Merge: Model Merging on the Right Space Improves Multitasking 29 May 2025 · 0 repositories · arXiv:2505.23117
-
ShIOEnv: A CLI Behavior-Capturing Environment Enabling Grammar-Guided Command Synthesis for Dataset Curation 23 May 2025 · 1 repository · arXiv:2505.18374
-
LogiCase: Effective Test Case Generation from Logical Description in Competitive Programming 21 May 2025 · 0 repositories · arXiv:2505.15039Syntology 11 ran (of which 3 constructed an object rather than computing a result; 5 with no instrument failure: 2 honoured, 0 violated, 3 with no contract checked; 6 where Syntology's instrument failed) · 8 unverified (of 19 harvested samples) · 19 pointer-only (licence)
-
EEG-to-Text Translation: A Model for Deciphering Human Brain Activity 20 May 2025 · 1 repository · arXiv:2505.13936
-
Masking in Multi-hop QA: An Analysis of How Language Models Perform with Context Permutation 16 May 2025 · 1 repository · arXiv:2505.11754
-
Multilingual Machine Translation with Quantum Encoder Decoder Attention-based Convolutional Variational Circuits 14 May 2025 · 0 repositories · arXiv:2505.09407
-
Performance Evaluation of Large Language Models in Bangla Consumer Health Query Summarization 8 May 2025 · 0 repositories · arXiv:2505.05070
-
GASCADE: Grouped Summarization of Adverse Drug Event for Enhanced Cancer Pharmacovigilance 7 May 2025 · 1 repository · arXiv:2505.04284
-
A review of DNA restriction-free overlapping sequence cloning techniques for synthetic biology 6 May 2025 · 0 repositories · arXiv:2505.03681
-
JaccDiv: A Metric and Benchmark for Quantifying Diversity of Generated Marketing Text in the Music Industry 29 Apr 2025 · 0 repositories · arXiv:2504.20849
-
Large Language Models are Qualified Benchmark Builders: Rebuilding Pre-Training Datasets for Advancing Code Intelligence Tasks 28 Apr 2025 · 0 repositories · arXiv:2504.19444
-
Sigma: A dataset for text-to-code semantic parsing with statistical analysis 5 Apr 2025 · 1 repository · arXiv:2504.04301
-
Advancing Sentiment Analysis in Tamil-English Code-Mixed Texts: Challenges and Transformer-Based Solutions 30 Mar 2025 · 0 repositories · arXiv:2503.23295
-
Enhancing Knowledge Graph Completion with Entity Neighborhood and Relation Context 29 Mar 2025 · 0 repositories · arXiv:2503.23205
-
Scaling Down Text Encoders of Text-to-Image Diffusion Models 25 Mar 2025 · 1 repository · arXiv:2503.19897
-
Exploring Training and Inference Scaling Laws in Generative Retrieval 24 Mar 2025 · 1 repository · arXiv:2503.18941
-
Predicting the Road Ahead: A Knowledge Graph based Foundation Model for Scene Understanding in Autonomous Driving 24 Mar 2025 · 0 repositories · arXiv:2503.18730
-
Enhancing Code LLM Training with Programmer Attention 19 Mar 2025 · 0 repositories · arXiv:2503.14936
-
DreamRenderer: Taming Multi-Instance Attribute Control in Large-Scale Text-to-Image Models 17 Mar 2025 · 0 repositories · arXiv:2503.12885
-
ARLED: Leveraging LED-based ARMAN Model for Abstractive Summarization of Persian Long Documents 13 Mar 2025 · 0 repositories · arXiv:2503.10233
-
A LongFormer-Based Framework for Accurate and Efficient Medical Text Summarization 10 Mar 2025 · 0 repositories · arXiv:2503.06888
-
Roamify: Designing and Evaluating an LLM Based Google Chrome Extension for Personalised Itinerary Planning 10 Mar 2025 · 1 repository · arXiv:2504.10489
-
Seeing Delta Parameters as JPEG Images: Data-Free Delta Compression with Discrete Cosine Transform 9 Mar 2025 · 0 repositories · arXiv:2503.06676
-
MoEMoE: Question Guided Dense and Scalable Sparse Mixture-of-Expert for Multi-source Multi-modal Answering 8 Mar 2025 · 0 repositories · arXiv:2503.06296
-
A Transformer Model for Predicting Chemical Reaction Products from Generic Templates 4 Mar 2025 · 0 repositories · arXiv:2503.05810
-
Multimodal Representation Alignment for Image Generation: Text-Image Interleaved Control Is Easier Than You Think 27 Feb 2025 · 1 repository · arXiv:2502.20172
-
LDGen: Enhancing Text-to-Image Synthesis via Large Language Model-Driven Language Representation 25 Feb 2025 · 0 repositories · arXiv:2502.18302
-
A Fine-Tuning Approach for T5 Using Knowledge Graphs to Address Complex Tasks 23 Feb 2025 · 0 repositories · arXiv:2502.16484
-
Corrections Meet Explanations: A Unified Framework for Explainable Grammatical Error Correction 21 Feb 2025 · 0 repositories · arXiv:2502.15261
-
Empowering LLMs with Logical Reasoning: A Comprehensive Survey 21 Feb 2025 · 0 repositories · arXiv:2502.15652
-
Bridging Text and Vision: A Multi-View Text-Vision Registration Approach for Cross-Modal Place Recognition 20 Feb 2025 · 1 repository · arXiv:2502.14195
-
Order-agnostic Identifier for Large Language Model-based Generative Recommendation 15 Feb 2025 · 0 repositories · arXiv:2502.10833
-
A Large-Scale Benchmark for Vietnamese Sentence Paraphrases 11 Feb 2025 · 1 repository · arXiv:2502.07188
-
Robust and Secure Code Watermarking for Large Language Models via ML/Crypto Codesign 4 Feb 2025 · 0 repositories · arXiv:2502.02068
-
General Embedding vs. Task-Specific Embedding: A Comparative Approach to Enhancing NLP Performance 30 Jan 2025 · 0 repositories
-
Visualizing Uncertainty in Translation Tasks: An Evaluation of LLM Performance and Confidence Metrics 26 Jan 2025 · 1 repository · arXiv:2501.17187
-
Diffusion based Text-to-Music Generation with Global and Local Text based Conditioning 24 Jan 2025 · 0 repositories · arXiv:2501.14680
-
Confidence Estimation for Error Detection in Text-to-SQL Systems 16 Jan 2025 · 1 repository · arXiv:2501.09527
-
Exploring Robustness of Multilingual LLMs on Real-World Noisy Data 14 Jan 2025 · 1 repository · arXiv:2501.08322
-
MathReader : Text-to-Speech for Mathematical Documents 13 Jan 2025 · 1 repository · arXiv:2501.07088
-
Bridging Dialects: Translating Standard Bangla to Regional Variants Using Neural Models 10 Jan 2025 · 0 repositories · arXiv:2501.05749
-
Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directions 8 Jan 2025 · 0 repositories · arXiv:2501.04437
-
How to Select Pre-Trained Code Models for Reuse? A Learning Perspective 7 Jan 2025 · 1 repository · arXiv:2501.03783
-
Empowering Bengali Education with AI: Solving Bengali Math Word Problems through Transformer Models 5 Jan 2025 · 0 repositories · arXiv:2501.02599
-
BeliN: A Novel Corpus for Bengali Religious News Headline Generation using Contextual Feature Fusion 2 Jan 2025 · 1 repository · arXiv:2501.01069
-
YAD: Leveraging T5 for Improved Automatic Diacritization of Yorùbá Text 28 Dec 2024 · 1 repository · arXiv:2412.20218
-
HAND: Hierarchical Attention Network for Multi-Scale Handwritten Document Recognition and Layout Analysis 25 Dec 2024 · 0 repositories · arXiv:2412.18981
-
Optimizing Large Language Models with an Enhanced LoRA Fine-Tuning Algorithm for Efficiency and Robustness in NLP Tasks 25 Dec 2024 · 0 repositories · arXiv:2412.18729
-
Advancing Explainability in Neural Machine Translation: Analytical Metrics for Attention and Alignment Consistency 24 Dec 2024 · 0 repositories · arXiv:2412.18669
-
HTR-JAND: Handwritten Text Recognition with Joint Attention Network and Knowledge Distillation 24 Dec 2024 · 0 repositories · arXiv:2412.18524
-
Identifying Cyberbullying Roles in Social Media 21 Dec 2024 · 0 repositories · arXiv:2412.16417
-
Adversarial Robustness through Dynamic Ensemble Learning 20 Dec 2024 · 0 repositories · arXiv:2412.16254
-
Can LLMs Obfuscate Code? A Systematic Analysis of Large Language Models into Assembly Code Obfuscation 20 Dec 2024 · 0 repositories · arXiv:2412.16135
-
The First Multilingual Model For The Detection of Suicide Texts 20 Dec 2024 · 0 repositories · arXiv:2412.15498
-
LLMCL-GEC: Advancing Grammatical Error Correction with LLM-Driven Curriculum Learning 17 Dec 2024 · 0 repositories · arXiv:2412.12541
-
Graph-Guided Textual Explanation Generation Framework 16 Dec 2024 · 0 repositories · arXiv:2412.12318
-
Neural Text Normalization for Luxembourgish using Real-Life Variation Data 12 Dec 2024 · 0 repositories · arXiv:2412.09383
-
Text Generation Models for Luxembourgish with Limited Data: A Balanced Multilingual Strategy 12 Dec 2024 · 0 repositories · arXiv:2412.09415
-
Are Clinical T5 Models Better for Clinical Text? 8 Dec 2024 · 1 repository · arXiv:2412.05845
-
Leveraging Large Language Models for Comparative Literature Summarization with Reflective Incremental Mechanisms 3 Dec 2024 · 0 repositories · arXiv:2412.02149
-
SiTSE: Sinhala Text Simplification Dataset and Evaluation 2 Dec 2024 · 1 repository · arXiv:2412.01293
-
A Comprehensive Guide to Explainable AI: From Classical Models to LLMs 1 Dec 2024 · 1 repository · arXiv:2412.00800
-
Towards Santali Linguistic Inclusion: Building the First Santali-to-English Translation Model using mT5 Transformer and Data Augmentation 29 Nov 2024 · 0 repositories · arXiv:2411.19726
-
An Extensive Evaluation of Factual Consistency in Large Language Models for Data-to-Text Generation 28 Nov 2024 · 0 repositories · arXiv:2411.19203
-
A survey on cutting-edge relation extraction techniques based on language models 27 Nov 2024 · 0 repositories · arXiv:2411.18157
-
Generative Fuzzy System for Sequence Generation 21 Nov 2024 · 0 repositories · arXiv:2411.13867
-
A Combined Encoder and Transformer Approach for Coherent and High-Quality Text Generation 19 Nov 2024 · 0 repositories · arXiv:2411.12157
-
Leveraging Multimodal Models for Enhanced Neuroimaging Diagnostics in Alzheimer's Disease 12 Nov 2024 · 0 repositories · arXiv:2411.07871
-
Regress, Don't Guess -- A Regression-like Loss on Number Tokens for Language Models 4 Nov 2024 · 1 repository · arXiv:2411.02083Syntology official (archive's flag): 1 ran · 1 ran (of which 1 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; the one sample that ran constructed an object rather than computing a result (of 1 harvested sample)
-
Training-free Regional Prompting for Diffusion Transformers 4 Nov 2024 · 1 repository · arXiv:2411.02395Syntology official (archive's flag): 1 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 2 pointer-only (licence)
-
ProTransformer: Robustify Transformers via Plug-and-Play Paradigm 30 Oct 2024 · 1 repository · arXiv:2410.23182
-
Building A Coding Assistant via the Retrieval-Augmented Language Model 21 Oct 2024 · 1 repository · arXiv:2410.16229
-
Unlocking Legal Knowledge: A Multilingual Dataset for Judicial Summarization in Switzerland 17 Oct 2024 · 0 repositories · arXiv:2410.13456
-
Tokenization and Morphology in Multilingual Language Models: A Comparative Analysis of mT5 and ByT5 15 Oct 2024 · 0 repositories · arXiv:2410.11627
-
SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformers 14 Oct 2024 · 2 repositories · arXiv:2410.10629
-
Single Ground Truth Is Not Enough: Add Linguistic Variability to Aspect-based Sentiment Analysis Evaluation 13 Oct 2024 · 0 repositories · arXiv:2410.09807
-
Enhancing SPARQL Generation by Triplet-order-sensitive Pre-training 8 Oct 2024 · 1 repository · arXiv:2410.05731
-
Leveraging free energy in pretraining model selection for improved fine-tuning 8 Oct 2024 · 0 repositories · arXiv:2410.05612
-
Deep Transfer Learning Based Peer Review Aggregation and Meta-review Generation for Scientific Articles 5 Oct 2024 · 0 repositories · arXiv:2410.04202
-
GPT-4o as the Gold Standard: A Scalable and General Purpose Approach to Filter Language Model Pretraining Data 3 Oct 2024 · 0 repositories · arXiv:2410.02755
-
Enhancing LLM Fine-tuning for Text-to-SQLs by SQL Quality Measurement 2 Oct 2024 · 0 repositories · arXiv:2410.01869
-
Abstractive Summarization of Low resourced Nepali language using Multilingual Transformers 29 Sep 2024 · 0 repositories · arXiv:2409.19566
-
Enhancing Automatic Keyphrase Labelling with Text-to-Text Transfer Transformer (T5) Architecture: A Framework for Keyphrase Generation and Filtering 25 Sep 2024 · 0 repositories · arXiv:2409.16760
-
Probing Omissions and Distortions in Transformer-based RDF-to-Text Models 25 Sep 2024 · 0 repositories · arXiv:2409.16707
-
Do the Right Thing, Just Debias! Multi-Category Bias Mitigation Using LLMs 24 Sep 2024 · 0 repositories · arXiv:2409.16371
-
Improving Academic Skills Assessment with NLP and Ensemble Learning 23 Sep 2024 · 0 repositories · arXiv:2409.19013
-
EMMeTT: Efficient Multimodal Machine Translation Training 20 Sep 2024 · 0 repositories · arXiv:2409.13523
-
Exploring Large Language Models for Product Attribute Value Identification 19 Sep 2024 · 0 repositories · arXiv:2409.12695
-
MURI: High-Quality Instruction Tuning Datasets for Low-Resource Languages via Reverse Instructions 19 Sep 2024 · 1 repository · arXiv:2409.12958
-
Small Language Models are Equation Reasoners 19 Sep 2024 · 0 repositories · arXiv:2409.12393
-
Chain-of-Thought Prompting for Speech Translation 17 Sep 2024 · 0 repositories · arXiv:2409.11538
-
Exploring Fine-tuned Generative Models for Keyphrase Selection: A Case Study for Russian 16 Sep 2024 · 0 repositories · arXiv:2409.10640
-
Playground v3: Improving Text-to-Image Alignment with Deep-Fusion Large Language Models 16 Sep 2024 · 1 repository · arXiv:2409.10695Syntology 18 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 4 honoured, 1 violated, 2 with no contract checked; 11 where Syntology's instrument failed) · 3 unverified (of 21 harvested samples) · 21 pointer-only (licence)
-
Detection Made Easy: Potentials of Large Language Models for Solidity Vulnerabilities 15 Sep 2024 · 0 repositories · arXiv:2409.10574
Tasks archive 2025-07-28
20 shown of 472 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections