{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/revisiting-deep-learning-models-for-tabular","title":"Revisiting Deep Learning Models for Tabular Data","arxiv_id":"2106.11959","date":"2021-06-22","proceeding":"NeurIPS 2021 12","authors":["Yury Gorishniy","Ivan Rubachev","Valentin Khrulkov","Artem Babenko"],"abstract":"The existing literature on deep learning for tabular data proposes a wide range of novel architectures and reports competitive results on various datasets. However, the proposed models are usually not properly compared to each other and existing works often use different benchmarks and experiment protocols. As a result, it is unclear for both researchers and practitioners what models perform best. Additionally, the field still lacks effective baselines, that is, the easy-to-use models that provide competitive performance across different problems. In this work, we perform an overview of the main families of DL architectures for tabular data and raise the bar of baselines in tabular DL by identifying two simple and powerful deep architectures. The first one is a ResNet-like architecture which turns out to be a strong baseline that is often missing in prior works. The second model is our simple adaptation of the Transformer architecture for tabular data, which outperforms other solutions on most tasks. Both models are compared to many existing architectures on a diverse set of tasks under the same training and tuning protocols. We also compare the best DL models with Gradient Boosted Decision Trees and conclude that there is still no universally superior solution.","url_abs":"https://arxiv.org/abs/2106.11959v5","url_pdf":"https://arxiv.org/pdf/2106.11959v5.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"revisiting-deep-learning-models-for-tabular","repo_url":"https://github.com/yandex-research/tabular-dl-revisiting-models","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"revisiting-deep-learning-models-for-tabular","repo_url":"https://github.com/Yura52/tabular-dl-revisiting-models","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"revisiting-deep-learning-models-for-tabular","repo_url":"https://github.com/XavierSpycy/tabtransformers","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"revisiting-deep-learning-models-for-tabular","repo_url":"https://github.com/avinashbarnwal/Deep-Learning","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"revisiting-deep-learning-models-for-tabular","repo_url":"https://github.com/axrwl/fttransformer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"revisiting-deep-learning-models-for-tabular","repo_url":"https://github.com/jrzaurin/pytorch-widedeep","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"revisiting-deep-learning-models-for-tabular","repo_url":"https://github.com/lucidrains/tab-transformer-pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"revisiting-deep-learning-models-for-tabular","repo_url":"https://github.com/manujosephv/pytorch_tabular","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"revisiting-deep-learning-models-for-tabular","repo_url":"https://github.com/yandex-research/rtdl-revisiting-models","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"revisiting-deep-learning-models-for-tabular","repo_url":"https://github.com/aruberts/TabTransformerTF","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"revisiting-deep-learning-models-for-tabular","repo_url":"https://github.com/kalelpark/DeepLearning-for-Tabular-Data","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"tabular-classification","task_name":"tabular-classification"},{"task_slug":"tabular-regression","task_name":"tabular-regression"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"ft-transformer","method_name":"FT-Transformer"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2106.11959","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2106.11959"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/yandex-research/tabular-dl-revisiting-models","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/aruberts/TabTransformerTF","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/XavierSpycy/tabtransformers","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kalelpark/DeepLearning-for-Tabular-Data","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/axrwl/fttransformer","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/jrzaurin/pytorch-widedeep","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/yandex-research/rtdl-revisiting-models","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Yura52/tabular-dl-revisiting-models","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lucidrains/tab-transformer-pytorch","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/manujosephv/pytorch_tabular","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/avinashbarnwal/Deep-Learning","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":3,"ran_violates":1,"unverified":18},"by_repo_kind":{"official":{"samples":13,"ran":3,"repositories":1},"listed":{"samples":9,"ran":1,"repositories":3}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5bbdefbbff4e08d8","entry":"FeedForward","repo":"lucidrains/tab-transformer-pytorch","repo_kind":"listed","path":"tab_transformer_pytorch/ft_transformer.py","file_url":"https://github.com/lucidrains/tab-transformer-pytorch/blob/HEAD/tab_transformer_pytorch/ft_transformer.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5bbdefbbff4e08d8"}},{"code_sha256_prefix":"d7bace13fb369137","entry":"geglu","repo":"Yura52/tabular-dl-revisiting-models","repo_kind":"official","path":"lib/deep.py","file_url":"https://github.com/Yura52/tabular-dl-revisiting-models/blob/HEAD/lib/deep.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d7bace13fb369137"}},{"code_sha256_prefix":"9e66eafce9e7d0c5","entry":"needs_wd","repo":"Yura52/tabular-dl-revisiting-models","repo_kind":"official","path":"bin/autoint.py","file_url":"https://github.com/Yura52/tabular-dl-revisiting-models/blob/HEAD/bin/autoint.py","link_basis":"plan_row","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9e66eafce9e7d0c5"}},{"code_sha256_prefix":"2fb665ac3ecd7c20","entry":"reglu","repo":"Yura52/tabular-dl-revisiting-models","repo_kind":"official","path":"lib/deep.py","file_url":"https://github.com/Yura52/tabular-dl-revisiting-models/blob/HEAD/lib/deep.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2fb665ac3ecd7c20"}},{"code_sha256_prefix":"12f448bfa8afc6b7","entry":"calculate_metrics","repo":"Yura52/tabular-dl-revisiting-models","repo_kind":"official","path":"lib/metrics.py","file_url":"https://github.com/Yura52/tabular-dl-revisiting-models/blob/HEAD/lib/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"12f448bfa8afc6b7"}},{"code_sha256_prefix":"4d711eaf9f0bb383","entry":"f1_score_macro","repo":"XavierSpycy/tabtransformers","repo_kind":"listed","path":"tabtransformers/metrics.py","file_url":"https://github.com/XavierSpycy/tabtransformers/blob/HEAD/tabtransformers/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4d711eaf9f0bb383"}},{"code_sha256_prefix":"755bddefe05fb03e","entry":"get_activation_fn","repo":"jrzaurin/pytorch-widedeep","repo_kind":"listed","path":"pytorch_widedeep/models/_get_activation_fn.py","file_url":"https://github.com/jrzaurin/pytorch-widedeep/blob/HEAD/pytorch_widedeep/models/_get_activation_fn.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"755bddefe05fb03e"}},{"code_sha256_prefix":"f8e126db3b9cc8e1","entry":"get_data","repo":"XavierSpycy/tabtransformers","repo_kind":"listed","path":"tabtransformers/tools.py","file_url":"https://github.com/XavierSpycy/tabtransformers/blob/HEAD/tabtransformers/tools.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f8e126db3b9cc8e1"}},{"code_sha256_prefix":"d29f63635e0b6468","entry":"get_data_loader","repo":"XavierSpycy/tabtransformers","repo_kind":"listed","path":"tabtransformers/tools.py","file_url":"https://github.com/XavierSpycy/tabtransformers/blob/HEAD/tabtransformers/tools.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d29f63635e0b6468"}},{"code_sha256_prefix":"516fa25431b82747","entry":"get_loss_function","repo":"XavierSpycy/tabtransformers","repo_kind":"listed","path":"tabtransformers/config.py","file_url":"https://github.com/XavierSpycy/tabtransformers/blob/HEAD/tabtransformers/config.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"516fa25431b82747"}},{"code_sha256_prefix":"6e2718cdfa77cc6a","entry":"get_model","repo":"XavierSpycy/tabtransformers","repo_kind":"listed","path":"tabtransformers/config.py","file_url":"https://github.com/XavierSpycy/tabtransformers/blob/HEAD/tabtransformers/config.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6e2718cdfa77cc6a"}},{"code_sha256_prefix":"e7ecc99fa574de72","entry":"glu","repo":"Yura52/tabular-dl-revisiting-models","repo_kind":"official","path":"bin/tabnet.py","file_url":"https://github.com/Yura52/tabular-dl-revisiting-models/blob/HEAD/bin/tabnet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e7ecc99fa574de72"}},{"code_sha256_prefix":"b71e1c227eac8b6f","entry":"init_gbnn","repo":"Yura52/tabular-dl-revisiting-models","repo_kind":"official","path":"bin/grownet.py","file_url":"https://github.com/Yura52/tabular-dl-revisiting-models/blob/HEAD/bin/grownet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b71e1c227eac8b6f"}},{"code_sha256_prefix":"d5ca06455ae6a9a5","entry":"load_config","repo":"XavierSpycy/tabtransformers","repo_kind":"listed","path":"tabtransformers/config.py","file_url":"https://github.com/XavierSpycy/tabtransformers/blob/HEAD/tabtransformers/config.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d5ca06455ae6a9a5"}},{"code_sha256_prefix":"657558d2e8d19a32","entry":"load_dataset_info","repo":"Yura52/tabular-dl-revisiting-models","repo_kind":"official","path":"lib/data.py","file_url":"https://github.com/Yura52/tabular-dl-revisiting-models/blob/HEAD/lib/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"657558d2e8d19a32"}},{"code_sha256_prefix":"2f33af03ea909beb","entry":"load_json","repo":"Yura52/tabular-dl-revisiting-models","repo_kind":"official","path":"lib/util.py","file_url":"https://github.com/Yura52/tabular-dl-revisiting-models/blob/HEAD/lib/util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2f33af03ea909beb"}},{"code_sha256_prefix":"ba8f59b208956c3b","entry":"load_pickle","repo":"Yura52/tabular-dl-revisiting-models","repo_kind":"official","path":"lib/util.py","file_url":"https://github.com/Yura52/tabular-dl-revisiting-models/blob/HEAD/lib/util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ba8f59b208956c3b"}},{"code_sha256_prefix":"27e9ee3019bc8ff0","entry":"make_optimizer","repo":"Yura52/tabular-dl-revisiting-models","repo_kind":"official","path":"lib/deep.py","file_url":"https://github.com/Yura52/tabular-dl-revisiting-models/blob/HEAD/lib/deep.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"27e9ee3019bc8ff0"}},{"code_sha256_prefix":"9fb7a99bb9386bb6","entry":"make_summary","repo":"Yura52/tabular-dl-revisiting-models","repo_kind":"official","path":"lib/metrics.py","file_url":"https://github.com/Yura52/tabular-dl-revisiting-models/blob/HEAD/lib/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9fb7a99bb9386bb6"}},{"code_sha256_prefix":"e9e943ad18a8b419","entry":"normalize","repo":"Yura52/tabular-dl-revisiting-models","repo_kind":"official","path":"lib/data.py","file_url":"https://github.com/Yura52/tabular-dl-revisiting-models/blob/HEAD/lib/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e9e943ad18a8b419"}},{"code_sha256_prefix":"a18290fdacb1226e","entry":"root_mean_squared_logarithmic_error","repo":"XavierSpycy/tabtransformers","repo_kind":"listed","path":"tabtransformers/metrics.py","file_url":"https://github.com/XavierSpycy/tabtransformers/blob/HEAD/tabtransformers/metrics.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a18290fdacb1226e"}},{"code_sha256_prefix":"33a6192b7abb8353","entry":"to_tensors","repo":"Yura52/tabular-dl-revisiting-models","repo_kind":"official","path":"lib/data.py","file_url":"https://github.com/Yura52/tabular-dl-revisiting-models/blob/HEAD/lib/data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"33a6192b7abb8353"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}