{"url":"/method/performer","slug":"performer","name":"Performer","full_name":"Performer","full_name_withheld":false,"description_markdown":"**Performer** is a [Transformer](https://paperswithcode.com/methods/category/transformers) architecture which can estimate regular ([softmax](https://paperswithcode.com/method/softmax)) full-rank-attention Transformers with provable accuracy, but using only linear (as opposed to quadratic) space and time complexity, without relying on any priors such as sparsity or low-rankness. Performers are linear architectures fully compatible with regular Transformers and with strong theoretical guarantees: unbiased or nearly-unbiased estimation of the attention matrix, uniform convergence and low estimation variance. To approximate softmax attention-kernels, Performers use a Fast Attention Via positive Orthogonal Random features approach (FAVOR+), leveraging new methods for approximating softmax and Gaussian kernels.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Rethinking Attention with Performers","paper":"/paper/rethinking-attention-with-performers","first_author":"Krzysztof Choromanski","n_authors":13,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/rethinking-attention-with-performers"},"source":{"url":"https://arxiv.org/abs/2009.14794v4","title":"Rethinking Attention with Performers","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Natural Language Processing","area_id":"natural-language-processing","collection":"Transformers","url":"/methods/category/transformers","pwc_aliases":[]}],"n_papers_tagged":103,"archive_num_papers":103,"papers_newest_first":[{"paper":null,"title":"ExpertLongBench: Benchmarking Language Models on Expert-Level Long-Form Generation Tasks with Structured Checklists","date":"2025-06-02","arxiv_id":"2506.01241","n_code_links":0,"syntology":null},{"paper":null,"title":"WirelessMathBench: A Mathematical Modeling Benchmark for LLMs in Wireless Communications","date":"2025-05-20","arxiv_id":"2505.14354","n_code_links":0,"syntology":null},{"paper":null,"title":"CacheFormer: High Attention-Based Segment Caching","date":"2025-04-18","arxiv_id":"2504.13981","n_code_links":0,"syntology":null},{"paper":null,"title":"Deconstructing Jazz Piano Style Using Machine Learning","date":"2025-04-07","arxiv_id":"2504.05009","n_code_links":0,"syntology":null},{"paper":"/paper/predicting-survivability-of-cancer-patients","title":"Predicting Survivability of Cancer Patients with Metastatic Patterns Using Explainable AI","date":"2025-04-07","arxiv_id":"2504.06306","n_code_links":1,"syntology":null},{"paper":null,"title":"Forecasting Empty Container availability for Vehicle Booking System Application","date":"2025-03-14","arxiv_id":"2503.11728","n_code_links":0,"syntology":null},{"paper":"/paper/stead-spatio-temporal-efficient-anomaly-1","title":"STEAD: Spatio-Temporal Efficient Anomaly Detection for Time and Compute Sensitive Applications","date":"2025-03-11","arxiv_id":"2503.07942","n_code_links":1,"syntology":null},{"paper":null,"title":"Deep Learning-Based Approach for Automatic 2D and 3D MRI Segmentation of Gliomas","date":"2025-02-27","arxiv_id":"2502.19760","n_code_links":0,"syntology":null},{"paper":null,"title":"On the use of Performer and Agent Attention for Spoken Language Identification","date":"2025-02-09","arxiv_id":"2502.05841","n_code_links":0,"syntology":null},{"paper":null,"title":"Nick Patrick Contreras","date":"2025-01-09","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"title":"Comparative Study of Deep Learning Architectures for Textual Damage Level Classification","date":"2025-01-03","arxiv_id":"2501.01694","n_code_links":0,"syntology":null},{"paper":"/paper/electra-and-gpt-4o-cost-effective-partners","title":"ELECTRA and GPT-4o: Cost-Effective Partners for Sentiment Analysis","date":"2024-12-29","arxiv_id":"2501.00062","n_code_links":1,"syntology":null},{"paper":null,"title":"Music Genre Classification: Ensemble Learning with Subcomponents-level Attention","date":"2024-12-20","arxiv_id":"2412.15602","n_code_links":0,"syntology":null},{"paper":null,"title":"The Two-Hop Curse: LLMs trained on A$\\rightarrow$B, B$\\rightarrow$C fail to learn A$\\rightarrow$C","date":"2024-11-25","arxiv_id":"2411.16353","n_code_links":0,"syntology":null},{"paper":null,"title":"SOAR: Self-Occluded Avatar Recovery from a Single Video In the Wild","date":"2024-10-31","arxiv_id":"2410.23800","n_code_links":0,"syntology":null},{"paper":null,"title":"PerTok: Expressive Encoding and Modeling of Symbolic Musical Ideas and Variations","date":"2024-10-02","arxiv_id":"2410.02060","n_code_links":0,"syntology":null},{"paper":null,"title":"GLMHA A Guided Low-rank Multi-Head Self-Attention for Efficient Image Restoration and Spectral Reconstruction","date":"2024-10-01","arxiv_id":"2410.00380","n_code_links":0,"syntology":null},{"paper":"/paper/large-language-models-versus-classical","title":"Large Language Models versus Classical Machine Learning: Performance in COVID-19 Mortality Prediction Using High-Dimensional Tabular Data","date":"2024-09-02","arxiv_id":"2409.02136","n_code_links":2,"syntology":null},{"paper":null,"title":"Equitable Skin Disease Prediction Using Transfer Learning and Domain Adaptation","date":"2024-09-01","arxiv_id":"2409.00873","n_code_links":0,"syntology":null},{"paper":null,"title":"Tangram: Benchmark for Evaluating Geometric Element Recognition in Large Multimodal Models","date":"2024-08-25","arxiv_id":"2408.13854","n_code_links":0,"syntology":null},{"paper":"/paper/gabinsight-exploring-gender-activity-binding","title":"GABInsight: Exploring Gender-Activity Binding Bias in Vision-Language Models","date":"2024-07-30","arxiv_id":"2407.21001","n_code_links":1,"syntology":null},{"paper":"/paper/motamot-a-dataset-for-revealing-the-supremacy","title":"Motamot: A Dataset for Revealing the Supremacy of Large Language Models over Transformer Models in Bengali Political Sentiment Analysis","date":"2024-07-28","arxiv_id":"2407.19528","n_code_links":1,"syntology":null},{"paper":null,"title":"scGHSOM: Hierarchical clustering and visualization of single-cell and CRISPR data using growing hierarchical SOM","date":"2024-07-24","arxiv_id":"2407.16984","n_code_links":0,"syntology":null},{"paper":"/paper/predicting-the-best-of-n-visual-trackers","title":"Predicting the Best of N Visual Trackers","date":"2024-07-22","arxiv_id":"2407.15707","n_code_links":1,"syntology":null},{"paper":null,"title":"Lightweight Large Language Model for Medication Enquiry: Med-Pal","date":"2024-07-02","arxiv_id":"2407.12822","n_code_links":0,"syntology":null},{"paper":null,"title":"Deep Reinforcement Learning Strategies in Finance: Insights into Asset Holding, Trading Behavior, and Purchase Diversity","date":"2024-06-29","arxiv_id":"2407.09557","n_code_links":0,"syntology":null},{"paper":null,"title":"Modeling the Real World with High-Density Visual Particle Dynamics","date":"2024-06-28","arxiv_id":"2406.19800","n_code_links":0,"syntology":null},{"paper":"/paper/hopping-too-late-exploring-the-limitations-of","title":"Hopping Too Late: Exploring the Limitations of Large Language Models on Multi-Hop Queries","date":"2024-06-18","arxiv_id":"2406.12775","n_code_links":1,"syntology":{"ran":10,"of":12,"unverified":2,"pointer_only":0}},{"paper":"/paper/smart-expert-system-large-language-models-as","title":"Adaptable and Reliable Text Classification using Large Language Models","date":"2024-05-17","arxiv_id":"2405.10523","n_code_links":1,"syntology":null},{"paper":null,"title":"Designing and Evaluating Dialogue LLMs for Co-Creative Improvised Theatre","date":"2024-05-11","arxiv_id":"2405.07111","n_code_links":0,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/language-modelling","name":"Language Modelling","papers":6},{"task":"/task/decoder","name":"Decoder","papers":5},{"task":"/task/time-series-1","name":"Time Series","papers":5},{"task":"/task/classification-1","name":"Classification","papers":4},{"task":"/task/language-modeling","name":"Language Modeling","papers":4},{"task":"/task/time-series","name":"Time Series Analysis","papers":4},{"task":"/task/anomaly-detection","name":"Anomaly Detection","papers":3},{"task":"/task/computational-efficiency","name":"Computational Efficiency","papers":3},{"task":"/task/decision-making","name":"Decision Making","papers":3},{"task":null,"name":"GPU","papers":3},{"task":"/task/image-classification","name":"Image Classification","papers":3},{"task":"/task/nerf","name":"NeRF","papers":3},{"task":"/task/novel-view-synthesis","name":"Novel View Synthesis","papers":3},{"task":"/task/representation-learning","name":"Representation Learning","papers":3},{"task":"/task/segmentation","name":"Segmentation","papers":3},{"task":"/task/sentiment-analysis","name":"Sentiment Analysis","papers":3},{"task":"/task/image-classification","name":"image-classification","papers":3},{"task":"/task/autonomous-driving","name":"Autonomous Driving","papers":2},{"task":"/task/benchmarking","name":"Benchmarking","papers":2},{"task":"/task/clustering","name":"Clustering","papers":2}],"tasks_shown":20,"n_tasks":149,"usage_by_year":[{"year":"2020","papers":3},{"year":"2021","papers":17},{"year":"2022","papers":24},{"year":"2023","papers":23},{"year":"2024","papers":25},{"year":"2025","papers":11}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/performer"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}