{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/interpretable-machine-learning-for-tabpfn","title":"Interpretable Machine Learning for TabPFN","arxiv_id":"2403.10923","date":"2024-03-16","proceeding":null,"authors":["David Rundel","Julius Kobialka","Constantin von Crailsheim","Matthias Feurer","Thomas Nagler","David Rügamer"],"abstract":"The recently developed Prior-Data Fitted Networks (PFNs) have shown very promising results for applications in low-data regimes. The TabPFN model, a special case of PFNs for tabular data, is able to achieve state-of-the-art performance on a variety of classification tasks while producing posterior predictive distributions in mere seconds by in-context learning without the need for learning parameters or hyperparameter tuning. This makes TabPFN a very attractive option for a wide range of domain applications. However, a major drawback of the method is its lack of interpretability. Therefore, we propose several adaptations of popular interpretability methods that we specifically design for TabPFN. By taking advantage of the unique properties of the model, our adaptations allow for more efficient computations than existing implementations. In particular, we show how in-context learning facilitates the estimation of Shapley values by avoiding approximate retraining and enables the use of Leave-One-Covariate-Out (LOCO) even when working with large-scale Transformers. In addition, we demonstrate how data valuation methods can be used to address scalability challenges of TabPFN. Our proposed methods are implemented in a package tabpfn_iml and made available at https://github.com/david-rundel/tabpfn_iml.","url_abs":"https://arxiv.org/abs/2403.10923v2","url_pdf":"https://arxiv.org/pdf/2403.10923v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"interpretable-machine-learning-for-tabpfn","repo_url":"https://github.com/david-rundel/tabpfn_iml","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"data-valuation","task_name":"Data Valuation"},{"task_slug":"in-context-learning","task_name":"In-Context Learning"},{"task_slug":"interpretable-machine-learning","task_name":"Interpretable Machine Learning"}],"methods":[{"method_slug":"tabpfn","method_name":"TABPFN"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2403.10923","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.10923"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/david-rundel/tabpfn_iml","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":8,"unverified":4},"by_repo_kind":{"official":{"samples":12,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"890fb6ef88c9dbef","entry":"JointBCELossWithLogits","repo":"david-rundel/tabpfn_iml","repo_kind":"official","path":"tabpfniml/tabpfn_interpret/losses.py","file_url":"https://github.com/david-rundel/tabpfn_iml/blob/HEAD/tabpfniml/tabpfn_interpret/losses.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"890fb6ef88c9dbef"}},{"code_sha256_prefix":"6751753e52ba3b27","entry":"draw_random_style","repo":"david-rundel/tabpfn_iml","repo_kind":"official","path":"tabpfniml/tabpfn_interpret/priors/differentiable_prior.py","file_url":"https://github.com/david-rundel/tabpfn_iml/blob/HEAD/tabpfniml/tabpfn_interpret/priors/differentiable_prior.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6751753e52ba3b27"}},{"code_sha256_prefix":"939b599ed6041748","entry":"get_append_embedding_encoder","repo":"david-rundel/tabpfn_iml","repo_kind":"official","path":"tabpfniml/tabpfn_interpret/encoders.py","file_url":"https://github.com/david-rundel/tabpfn_iml/blob/HEAD/tabpfniml/tabpfn_interpret/encoders.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"939b599ed6041748"}},{"code_sha256_prefix":"f72c39b3934d20d3","entry":"get_model","repo":"david-rundel/tabpfn_iml","repo_kind":"official","path":"tabpfniml/tabpfn_interpret/priors/fast_gp.py","file_url":"https://github.com/david-rundel/tabpfn_iml/blob/HEAD/tabpfniml/tabpfn_interpret/priors/fast_gp.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f72c39b3934d20d3"}},{"code_sha256_prefix":"cd13023248585727","entry":"get_model_on_device","repo":"david-rundel/tabpfn_iml","repo_kind":"official","path":"tabpfniml/tabpfn_interpret/priors/fast_gp.py","file_url":"https://github.com/david-rundel/tabpfn_iml/blob/HEAD/tabpfniml/tabpfn_interpret/priors/fast_gp.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cd13023248585727"}},{"code_sha256_prefix":"39f5dc76953cf6db","entry":"get_normalized_encoder","repo":"david-rundel/tabpfn_iml","repo_kind":"official","path":"tabpfniml/tabpfn_interpret/encoders.py","file_url":"https://github.com/david-rundel/tabpfn_iml/blob/HEAD/tabpfniml/tabpfn_interpret/encoders.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"39f5dc76953cf6db"}},{"code_sha256_prefix":"aa7ad3fc695f98b5","entry":"get_restarting_cosine_schedule_with_warmup","repo":"david-rundel/tabpfn_iml","repo_kind":"official","path":"tabpfniml/tabpfn_interpret/utils.py","file_url":"https://github.com/david-rundel/tabpfn_iml/blob/HEAD/tabpfniml/tabpfn_interpret/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa7ad3fc695f98b5"}},{"code_sha256_prefix":"a20e00970d0cbf4c","entry":"unpack_dict_of_tuples","repo":"david-rundel/tabpfn_iml","repo_kind":"official","path":"tabpfniml/tabpfn_interpret/priors/differentiable_prior.py","file_url":"https://github.com/david-rundel/tabpfn_iml/blob/HEAD/tabpfniml/tabpfn_interpret/priors/differentiable_prior.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a20e00970d0cbf4c"}},{"code_sha256_prefix":"4c84ce37831d7194","entry":"get_NormalInitializer","repo":"david-rundel/tabpfn_iml","repo_kind":"official","path":"tabpfniml/tabpfn_interpret/initializers.py","file_url":"https://github.com/david-rundel/tabpfn_iml/blob/HEAD/tabpfniml/tabpfn_interpret/initializers.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4c84ce37831d7194"}},{"code_sha256_prefix":"a9c14b9fb71dbe7c","entry":"get_cosine_schedule_with_warmup","repo":"david-rundel/tabpfn_iml","repo_kind":"official","path":"tabpfniml/tabpfn_interpret/utils.py","file_url":"https://github.com/david-rundel/tabpfn_iml/blob/HEAD/tabpfniml/tabpfn_interpret/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a9c14b9fb71dbe7c"}},{"code_sha256_prefix":"afba2c06a42074fe","entry":"get_linear_schedule_with_warmup","repo":"david-rundel/tabpfn_iml","repo_kind":"official","path":"tabpfniml/tabpfn_interpret/utils.py","file_url":"https://github.com/david-rundel/tabpfn_iml/blob/HEAD/tabpfniml/tabpfn_interpret/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"afba2c06a42074fe"}},{"code_sha256_prefix":"16eab2bd7416886c","entry":"get_normalized_uniform_encoder","repo":"david-rundel/tabpfn_iml","repo_kind":"official","path":"tabpfniml/tabpfn_interpret/encoders.py","file_url":"https://github.com/david-rundel/tabpfn_iml/blob/HEAD/tabpfniml/tabpfn_interpret/encoders.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"16eab2bd7416886c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}