{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/tabularbench-benchmarking-adversarial","title":"TabularBench: Benchmarking Adversarial Robustness for Tabular Deep Learning in Real-world Use-cases","arxiv_id":"2408.07579","date":"2024-08-14","proceeding":null,"authors":["Thibault Simonetto","Salah Ghamizi","Maxime Cordy"],"abstract":"While adversarial robustness in computer vision is a mature research field, fewer researchers have tackled the evasion attacks against tabular deep learning, and even fewer investigated robustification mechanisms and reliable defenses. We hypothesize that this lag in the research on tabular adversarial attacks is in part due to the lack of standardized benchmarks. To fill this gap, we propose TabularBench, the first comprehensive benchmark of robustness of tabular deep learning classification models. We evaluated adversarial robustness with CAA, an ensemble of gradient and search attacks which was recently demonstrated as the most effective attack against a tabular model. In addition to our open benchmark (https://github.com/serval-uni-lu/tabularbench) where we welcome submissions of new models and defenses, we implement 7 robustification mechanisms inspired by state-of-the-art defenses in computer vision and propose the largest benchmark of robust tabular deep learning over 200 models across five critical scenarios in finance, healthcare and security. We curated real datasets for each use case, augmented with hundreds of thousands of realistic synthetic inputs, and trained and assessed our models with and without data augmentations. We open-source our library that provides API access to all our pre-trained robust tabular models, and the largest datasets of real and synthetic tabular inputs. Finally, we analyze the impact of various defenses on the robustness and provide actionable insights to design new defenses and robustification mechanisms.","url_abs":"https://arxiv.org/abs/2408.07579v1","url_pdf":"https://arxiv.org/pdf/2408.07579v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"tabularbench-benchmarking-adversarial","repo_url":"https://github.com/serval-uni-lu/tabularbench","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"adversarial-robustness","task_name":"Adversarial Robustness"},{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"deep-learning","task_name":"Deep Learning"}],"methods":[{"method_slug":null,"method_name":"Library"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2408.07579","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2408.07579"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/serval-uni-lu/tabularbench","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":6},"by_repo_kind":{"official":{"samples":6,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"45a876757e98fd6e","entry":"cut_in_batch","repo":"serval-uni-lu/tabularbench","repo_kind":"official","path":"tabularbench/attacks/utils.py","file_url":"https://github.com/serval-uni-lu/tabularbench/blob/HEAD/tabularbench/attacks/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"45a876757e98fd6e"}},{"code_sha256_prefix":"1abf58446eee3e4c","entry":"fix_types","repo":"serval-uni-lu/tabularbench","repo_kind":"official","path":"tabularbench/attacks/utils.py","file_url":"https://github.com/serval-uni-lu/tabularbench/blob/HEAD/tabularbench/attacks/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1abf58446eee3e4c"}},{"code_sha256_prefix":"9439dfdb18198cd9","entry":"get_cat_idx","repo":"serval-uni-lu/tabularbench","repo_kind":"official","path":"tabularbench/models/tab_scaler.py","file_url":"https://github.com/serval-uni-lu/tabularbench/blob/HEAD/tabularbench/models/tab_scaler.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9439dfdb18198cd9"}},{"code_sha256_prefix":"59969863caa15237","entry":"remove_g_from_loss","repo":"serval-uni-lu/tabularbench","repo_kind":"official","path":"tabularbench/models/torch_models.py","file_url":"https://github.com/serval-uni-lu/tabularbench/blob/HEAD/tabularbench/models/torch_models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"59969863caa15237"}},{"code_sha256_prefix":"df25a2473c298d3a","entry":"round_func_BPDA","repo":"serval-uni-lu/tabularbench","repo_kind":"official","path":"tabularbench/models/tab_scaler.py","file_url":"https://github.com/serval-uni-lu/tabularbench/blob/HEAD/tabularbench/models/tab_scaler.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"df25a2473c298d3a"}},{"code_sha256_prefix":"6f9a297a15a8f406","entry":"softargmax","repo":"serval-uni-lu/tabularbench","repo_kind":"official","path":"tabularbench/models/tab_scaler.py","file_url":"https://github.com/serval-uni-lu/tabularbench/blob/HEAD/tabularbench/models/tab_scaler.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6f9a297a15a8f406"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}