{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/better-by-default-strong-pre-tuned-mlps-and","title":"Better by Default: Strong Pre-Tuned MLPs and Boosted Trees on Tabular Data","arxiv_id":"2407.04491","date":"2024-07-05","proceeding":null,"authors":["David Holzmüller","Léo Grinsztajn","Ingo Steinwart"],"abstract":"For classification and regression on tabular data, the dominance of gradient-boosted decision trees (GBDTs) has recently been challenged by often much slower deep learning methods with extensive hyperparameter tuning. We address this discrepancy by introducing (a) RealMLP, an improved multilayer perceptron (MLP), and (b) strong meta-tuned default parameters for GBDTs and RealMLP. We tune RealMLP and the default parameters on a meta-train benchmark with 118 datasets and compare them to hyperparameter-optimized versions on a disjoint meta-test benchmark with 90 datasets, as well as the GBDT-friendly benchmark by Grinsztajn et al. (2022). Our benchmark results on medium-to-large tabular datasets (1K--500K samples) show that RealMLP offers a favorable time-accuracy tradeoff compared to other neural baselines and is competitive with GBDTs in terms of benchmark scores. Moreover, a combination of RealMLP and GBDTs with improved default parameters can achieve excellent results without hyperparameter tuning. Finally, we demonstrate that some of RealMLP's improvements can also considerably improve the performance of TabR with default parameters.","url_abs":"https://arxiv.org/abs/2407.04491v3","url_pdf":"https://arxiv.org/pdf/2407.04491v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"better-by-default-strong-pre-tuned-mlps-and","repo_url":"https://github.com/dholzmueller/pytabkit","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"regression-1","task_name":"regression"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2407.04491","atlas_url":"https://app.syntology.ai/?focus=2407.04491","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.04491"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/LeoGrin/tabular-benchmark","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dholzmueller/pytabkit","reach":null}],"summary":{"ran":1,"unverified":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1},"found_in_text":{"samples":1,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"58ef450dbc562cfe","entry":"MLP","repo":"dholzmueller/pytabkit","repo_kind":"official","path":"pytabkit/models/nn_models/tabm.py","file_url":"https://github.com/dholzmueller/pytabkit/blob/HEAD/pytabkit/models/nn_models/tabm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"58ef450dbc562cfe"}},{"code_sha256_prefix":"636761905eb29852","entry":"MLP","repo":"LeoGrin/tabular-benchmark","repo_kind":"found_in_text","path":"src/models/tabular/bin/mlp.py","file_url":"https://github.com/LeoGrin/tabular-benchmark/blob/HEAD/src/models/tabular/bin/mlp.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"636761905eb29852"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}