{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2509-11950","title":"TabStruct: Measuring Structural Fidelity of Tabular Data","arxiv_id":"2509.11950","date":"2025-09-15","proceeding":"ICLR","authors":["Xiangjian Jiang","Nikola Simidjievski","Mateja Jamnik"],"abstract":"Evaluating tabular generators remains a challenging problem, as the unique causal structural prior of heterogeneous tabular data does not lend itself to intuitive human inspection. Recent work has introduced structural fidelity as a tabular-specific evaluation dimension to assess whether synthetic data complies with the causal structures of real data. However, existing benchmarks often neglect the interplay between structural fidelity and conventional evaluation dimensions, thus failing to provide a holistic understanding of model performance. Moreover, they are typically limited to toy datasets, as quantifying existing structural fidelity metrics requires access to ground-truth causal structures, which are rarely available for real-world datasets. In this paper, we propose a novel evaluation framework that jointly considers structural fidelity and conventional evaluation dimensions. We introduce a new evaluation metric, $\\textbf{global utility}$, which enables the assessment of structural fidelity even in the absence of ground-truth causal structures. In addition, we present $\\textbf{TabStruct}$, a comprehensive evaluation benchmark offering large-scale quantitative analysis on 13 tabular generators from nine distinct categories, across 29 datasets. Our results demonstrate that global utility provides a task-independent, domain-agnostic lens for tabular generator performance. We release the TabStruct benchmark suite, including all datasets, evaluation pipelines, and raw results. Code is available at https://github.com/SilenceX12138/TabStruct.","url_abs":"https://arxiv.org/abs/2509.11950","url_pdf":"https://arxiv.org/pdf/2509.11950","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2509.11950","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2509.11950"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/automl/TabPFN","reach":{"status":"ok","spdx":"Apache-2.0"}},{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/SilenceX12138/TabStruct","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":1,"unverified":7},"by_repo_kind":{"found_in_text":{"samples":8,"ran":1,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"af729535b3619185","entry":"is_oom_error","repo":"automl/TabPFN","repo_kind":"found_in_text","path":"src/tabpfn/errors.py","file_url":"https://github.com/automl/TabPFN/blob/HEAD/src/tabpfn/errors.py","link_basis":"plan_row","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"af729535b3619185"}},{"code_sha256_prefix":"bd71e97cc121a6c2","entry":"add_runtime_setup","repo":"SilenceX12138/TabStruct","repo_kind":"found_in_text","path":"src/tabstruct/common/runtime/config/argument.py","file_url":"https://github.com/SilenceX12138/TabStruct/blob/HEAD/src/tabstruct/common/runtime/config/argument.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bd71e97cc121a6c2"}},{"code_sha256_prefix":"2264768af8bea3a5","entry":"add_wandb_setup","repo":"SilenceX12138/TabStruct","repo_kind":"found_in_text","path":"src/tabstruct/common/runtime/config/argument.py","file_url":"https://github.com/SilenceX12138/TabStruct/blob/HEAD/src/tabstruct/common/runtime/config/argument.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2264768af8bea3a5"}},{"code_sha256_prefix":"26dc590421ad649f","entry":"check_license_accepted","repo":"automl/TabPFN","repo_kind":"found_in_text","path":"src/tabpfn/browser_auth.py","file_url":"https://github.com/automl/TabPFN/blob/HEAD/src/tabpfn/browser_auth.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"26dc590421ad649f"}},{"code_sha256_prefix":"318535e795f4b430","entry":"compute_metric_to_minimize","repo":"automl/TabPFN","repo_kind":"found_in_text","path":"src/tabpfn/inference_tuning.py","file_url":"https://github.com/automl/TabPFN/blob/HEAD/src/tabpfn/inference_tuning.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"318535e795f4b430"}},{"code_sha256_prefix":"982527453e82549b","entry":"parse_arguments","repo":"SilenceX12138/TabStruct","repo_kind":"found_in_text","path":"src/tabstruct/common/runtime/config/argument.py","file_url":"https://github.com/SilenceX12138/TabStruct/blob/HEAD/src/tabstruct/common/runtime/config/argument.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"982527453e82549b"}},{"code_sha256_prefix":"d0ce318482f5b776","entry":"try_browser_login","repo":"automl/TabPFN","repo_kind":"found_in_text","path":"src/tabpfn/browser_auth.py","file_url":"https://github.com/automl/TabPFN/blob/HEAD/src/tabpfn/browser_auth.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d0ce318482f5b776"}},{"code_sha256_prefix":"f1719a2523736e98","entry":"verify_token","repo":"automl/TabPFN","repo_kind":"found_in_text","path":"src/tabpfn/browser_auth.py","file_url":"https://github.com/automl/TabPFN/blob/HEAD/src/tabpfn/browser_auth.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f1719a2523736e98"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_api"},"syntology_extracted_results":null}