{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-data-centric-perspective-on-evaluating","title":"A Data-Centric Perspective on Evaluating Machine Learning Models for Tabular Data","arxiv_id":"2407.02112","date":"2024-07-02","proceeding":null,"authors":["Andrej Tschalzev","Sascha Marton","Stefan Lüdtke","Christian Bartelt","Heiner Stuckenschmidt"],"abstract":"Tabular data is prevalent in real-world machine learning applications, and new models for supervised learning of tabular data are frequently proposed. Comparative studies assessing the performance of models typically consist of model-centric evaluation setups with overly standardized data preprocessing. This paper demonstrates that such model-centric evaluations are biased, as real-world modeling pipelines often require dataset-specific preprocessing and feature engineering. Therefore, we propose a data-centric evaluation framework. We select 10 relevant datasets from Kaggle competitions and implement expert-level preprocessing pipelines for each dataset. We conduct experiments with different preprocessing pipelines and hyperparameter optimization (HPO) regimes to quantify the impact of model selection, HPO, feature engineering, and test-time adaptation. Our main findings are: 1. After dataset-specific feature engineering, model rankings change considerably, performance differences decrease, and the importance of model selection reduces. 2. Recent models, despite their measurable progress, still significantly benefit from manual feature engineering. This holds true for both tree-based models and neural networks. 3. While tabular data is typically considered static, samples are often collected over time, and adapting to distribution shifts can be important even in supposedly static data. These insights suggest that research efforts should be directed toward a data-centric perspective, acknowledging that tabular data requires feature engineering and often exhibits temporal characteristics. Our framework is available under: https://github.com/atschalz/dc_tabeval.","url_abs":"https://arxiv.org/abs/2407.02112v3","url_pdf":"https://arxiv.org/pdf/2407.02112v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-data-centric-perspective-on-evaluating","repo_url":"https://github.com/atschalz/dc_tabeval","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"feature-engineering","task_name":"Feature Engineering"},{"task_slug":"hyperparameter-optimization","task_name":"Hyperparameter Optimization"},{"task_slug":"model-selection","task_name":"Model Selection"},{"task_slug":"test-time-adaptation","task_name":"Test-time Adaptation"}],"methods":[{"method_slug":"hpo","method_name":"HPO"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.02112","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.02112"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/atschalz/dc_tabeval","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":3,"unverified":3},"by_repo_kind":{"official":{"samples":6,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8134a23b12463d94","entry":"entmax15TF","repo":"atschalz/dc_tabeval","repo_kind":"official","path":"GRANDE/GRANDE.py","file_url":"https://github.com/atschalz/dc_tabeval/blob/HEAD/GRANDE/GRANDE.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8134a23b12463d94"}},{"code_sha256_prefix":"53162b711345d934","entry":"get_metric","repo":"atschalz/dc_tabeval","repo_kind":"official","path":"utils.py","file_url":"https://github.com/atschalz/dc_tabeval/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"53162b711345d934"}},{"code_sha256_prefix":"d494ce11c4822d74","entry":"gini","repo":"atschalz/dc_tabeval","repo_kind":"official","path":"utils.py","file_url":"https://github.com/atschalz/dc_tabeval/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d494ce11c4822d74"}},{"code_sha256_prefix":"4c4df1ddeb1b8a5b","entry":"gather_over_axisTF","repo":"atschalz/dc_tabeval","repo_kind":"official","path":"GRANDE/GRANDE.py","file_url":"https://github.com/atschalz/dc_tabeval/blob/HEAD/GRANDE/GRANDE.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4c4df1ddeb1b8a5b"}},{"code_sha256_prefix":"de9d681a2c0eac64","entry":"multilabel_log_loss","repo":"atschalz/dc_tabeval","repo_kind":"official","path":"utils.py","file_url":"https://github.com/atschalz/dc_tabeval/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"de9d681a2c0eac64"}},{"code_sha256_prefix":"bc8c43095129e3c3","entry":"top_k_over_axisTF","repo":"atschalz/dc_tabeval","repo_kind":"official","path":"GRANDE/GRANDE.py","file_url":"https://github.com/atschalz/dc_tabeval/blob/HEAD/GRANDE/GRANDE.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bc8c43095129e3c3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}