{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/contexttab-a-semantics-aware-tabular-in","title":"ConTextTab: A Semantics-Aware Tabular In-Context Learner","arxiv_id":"2506.10707","date":"2025-06-12","proceeding":null,"authors":["Marco Spinaci","Marek Polewczyk","Maximilian Schambach","Sam Thelin"],"abstract":"Tabular in-context learning (ICL) has recently achieved state-of-the-art (SOTA) performance on several tabular prediction tasks. Previously restricted to classification problems on small tables, recent advances such as TabPFN and TabICL have extended its use to larger datasets. While being architecturally efficient and well-adapted to tabular data structures, current table-native ICL architectures, being trained exclusively on synthetic data, do not fully leverage the rich semantics and world knowledge contained in real-world tabular data. On another end of this spectrum, tabular ICL models based on pretrained large language models such as TabuLa-8B integrate deep semantic understanding and world knowledge but are only able to make use of a small amount of context due to inherent architectural limitations. With the aim to combine the best of both these worlds, we introduce ConTextTab, integrating semantic understanding and alignment into a table-native ICL framework. By employing specialized embeddings for different data modalities and by training on large-scale real-world tabular data, our model is competitive with SOTA across a broad set of benchmarks while setting a new standard on the semantically rich CARTE benchmark.","url_abs":"https://arxiv.org/abs/2506.10707v1","url_pdf":"https://arxiv.org/pdf/2506.10707v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"contexttab-a-semantics-aware-tabular-in","repo_url":"https://github.com/sap-samples/contexttab","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"in-context-learning","task_name":"In-Context Learning"},{"task_slug":"world-knowledge","task_name":"World Knowledge"}],"methods":[{"method_slug":"set","method_name":"SET"},{"method_slug":"tabpfn","method_name":"TABPFN"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2506.10707","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2506.10707"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/SAP-samples/sap-rpt-1-oss","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sap-samples/contexttab","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":2,"unverified":4},"by_repo_kind":{"official":{"samples":1,"ran":0,"repositories":1},"found_in_text":{"samples":5,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ce325afd5ae0c1f9","entry":"DateEmbeddings","repo":"SAP-samples/sap-rpt-1-oss","repo_kind":"found_in_text","path":"sap_rpt_oss/model/embeddings.py","file_url":"https://github.com/SAP-samples/sap-rpt-1-oss/blob/HEAD/sap_rpt_oss/model/embeddings.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ce325afd5ae0c1f9"}},{"code_sha256_prefix":"817fcef39bf4af00","entry":"LRU_Cache","repo":"SAP-samples/sap-rpt-1-oss","repo_kind":"found_in_text","path":"sap_rpt_oss/model/embeddings.py","file_url":"https://github.com/SAP-samples/sap-rpt-1-oss/blob/HEAD/sap_rpt_oss/model/embeddings.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"817fcef39bf4af00"}},{"code_sha256_prefix":"45767b563e510eee","entry":"CellEmbeddings","repo":"SAP-samples/sap-rpt-1-oss","repo_kind":"found_in_text","path":"sap_rpt_oss/model/embeddings.py","file_url":"https://github.com/SAP-samples/sap-rpt-1-oss/blob/HEAD/sap_rpt_oss/model/embeddings.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"45767b563e510eee"}},{"code_sha256_prefix":"bcc26df93a6dc665","entry":"SentenceEmbedder","repo":"SAP-samples/sap-rpt-1-oss","repo_kind":"found_in_text","path":"sap_rpt_oss/model/embeddings.py","file_url":"https://github.com/SAP-samples/sap-rpt-1-oss/blob/HEAD/sap_rpt_oss/model/embeddings.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bcc26df93a6dc665"}},{"code_sha256_prefix":"2abe05cf0d9e528a","entry":"Tokenizer","repo":"SAP-samples/sap-rpt-1-oss","repo_kind":"found_in_text","path":"sap_rpt_oss/model/embeddings.py","file_url":"https://github.com/SAP-samples/sap-rpt-1-oss/blob/HEAD/sap_rpt_oss/model/embeddings.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2abe05cf0d9e528a"}},{"code_sha256_prefix":"b7cee542c204e2d6","entry":"to_device","repo":"sap-samples/contexttab","repo_kind":"official","path":"sap_rpt_oss/rpt.py","file_url":"https://github.com/sap-samples/contexttab/blob/HEAD/sap_rpt_oss/rpt.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b7cee542c204e2d6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}