{"url":"/method/ft-transformer","slug":"ft-transformer","name":"FT-Transformer","full_name":"FT-Transformer","full_name_withheld":false,"description_markdown":"FT-Transformer (Feature Tokenizer + Transformer) is a simple adaptation of the [Transformer](/method/transformer) architecture for the tabular domain. The model (Feature Tokenizer component) transforms all features (categorical and numerical) to tokens and runs a stack of Transformer layers over the tokens, so every Transformer layer operates on the feature level of one object. (This model is similar to [AutoInt](/method/autoint)). In the Transformer component, the `[CLS]` token is appended to $T$. Then $L$ Transformer layers are applied. PreNorm is used for easier optimization and good performance. The final representation of the `[CLS]` token is used for prediction.","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"https://arxiv.org/abs/2106.11959v5","title":"Revisiting Deep Learning Models for Tabular Data","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Deep Tabular Learning","url":"/methods/category/deep-tabular-learning","pwc_aliases":[]}],"n_papers_tagged":10,"archive_num_papers":null,"papers_newest_first":[{"paper":null,"title":"LLM Embeddings for Deep Learning on Tabular Data","date":"2025-02-17","arxiv_id":"2502.11596","n_code_links":0,"syntology":null},{"paper":null,"title":"Representation Learning on Out of Distribution in Tabular Data","date":"2025-02-14","arxiv_id":"2502.10095","n_code_links":0,"syntology":null},{"paper":"/paper/application-of-tabular-transformer","title":"Application of Tabular Transformer Architectures for Operating System Fingerprinting","date":"2025-02-13","arxiv_id":"2502.09084","n_code_links":1,"syntology":null},{"paper":null,"title":"A Survey on Deep Tabular Learning","date":"2024-10-15","arxiv_id":"2410.12034","n_code_links":0,"syntology":null},{"paper":null,"title":"Efficient Feature Interactions with Transformers: Improving User Spending Propensity Predictions in Gaming","date":"2024-09-25","arxiv_id":"2409.17077","n_code_links":0,"syntology":null},{"paper":null,"title":"Mamba for Scalable and Efficient Personalized Recommendations","date":"2024-09-11","arxiv_id":"2409.17165","n_code_links":0,"syntology":null},{"paper":null,"title":"Cross-Table Pretraining towards a Universal Function Space for Heterogeneous Tabular Data","date":"2024-06-01","arxiv_id":"2406.00281","n_code_links":0,"syntology":null},{"paper":null,"title":"Self-supervised learning of multi-omics embeddings in the low-label, high-data regime","date":"2023-11-16","arxiv_id":"2311.09962","n_code_links":0,"syntology":null},{"paper":"/paper/xtab-cross-table-pretraining-for-tabular","title":"XTab: Cross-table Pretraining for Tabular Transformers","date":"2023-05-10","arxiv_id":"2305.06090","n_code_links":1,"syntology":{"ran":1,"of":1,"unverified":0,"pointer_only":1}},{"paper":"/paper/revisiting-deep-learning-models-for-tabular","title":"Revisiting Deep Learning Models for Tabular Data","date":"2021-06-22","arxiv_id":"2106.11959","n_code_links":11,"syntology":{"ran":4,"of":22,"unverified":18,"pointer_only":0}}],"papers_shown":10,"tasks":[{"task":"/task/deep-learning","name":"Deep Learning","papers":2},{"task":"/task/self-supervised-learning","name":"Self-Supervised Learning","papers":2},{"task":"/task/automl","name":"AutoML","papers":1},{"task":null,"name":"CPU","papers":1},{"task":"/task/computational-efficiency","name":"Computational Efficiency","papers":1},{"task":"/task/contrastive-learning","name":"Contrastive Learning","papers":1},{"task":"/task/denoising","name":"Denoising","papers":1},{"task":"/task/federated-learning","name":"Federated Learning","papers":1},{"task":"/task/mamba","name":"Mamba","papers":1},{"task":"/task/management","name":"Management","papers":1},{"task":"/task/music-recommendation","name":"Music Recommendation","papers":1},{"task":"/task/recommendation-systems","name":"Recommendation Systems","papers":1},{"task":"/task/representation-learning","name":"Representation Learning","papers":1},{"task":"/task/state-space-models","name":"State Space Models","papers":1},{"task":"/task/survey","name":"Survey","papers":1},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":1},{"task":"/task/feature-selection","name":"feature selection","papers":1},{"task":"/task/tabular-classification","name":"tabular-classification","papers":1},{"task":"/task/tabular-regression","name":"tabular-regression","papers":1}],"tasks_shown":19,"n_tasks":19,"usage_by_year":[{"year":"2021","papers":1},{"year":"2023","papers":2},{"year":"2024","papers":4},{"year":"2025","papers":3}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/ft-transformer"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}