{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-bayesian-approach-to-analysing-training-1","title":"A Bayesian Approach To Analysing Training Data Attribution In Deep Learning","arxiv_id":"2305.19765","date":"2023-05-31","proceeding":"NeurIPS 2023 11","authors":["Elisa Nguyen","Minjoon Seo","Seong Joon Oh"],"abstract":"Training data attribution (TDA) techniques find influential training data for the model's prediction on the test data of interest. They approximate the impact of down- or up-weighting a particular training sample. While conceptually useful, they are hardly applicable to deep models in practice, particularly because of their sensitivity to different model initialisation. In this paper, we introduce a Bayesian perspective on the TDA task, where the learned model is treated as a Bayesian posterior and the TDA estimates as random variables. From this novel viewpoint, we observe that the influence of an individual training sample is often overshadowed by the noise stemming from model initialisation and SGD batch composition. Based on this observation, we argue that TDA can only be reliably used for explaining deep model predictions that are consistently influenced by certain training data, independent of other noise factors. Our experiments demonstrate the rarity of such noise-independent training-test data pairs but confirm their existence. We recommend that future researchers and practitioners trust TDA estimates only in such cases. Further, we find a disagreement between ground truth and estimated TDA distributions and encourage future work to study this gap. Code is provided at https://github.com/ElisaNguyen/bayesian-tda.","url_abs":"https://arxiv.org/abs/2305.19765v2","url_pdf":"https://arxiv.org/pdf/2305.19765v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-bayesian-approach-to-analysing-training-1","repo_url":"https://github.com/elisanguyen/bayesian-tda","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"sgd","method_name":"SGD"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2305.19765","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2305.19765"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/elisanguyen/bayesian-tda","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":5},"by_repo_kind":{"official":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"6178188047b6f48c","entry":"compute_gradients","repo":"elisanguyen/bayesian-tda","repo_kind":"official","path":"nn_influence_utils.py","file_url":"https://github.com/elisanguyen/bayesian-tda/blob/HEAD/nn_influence_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6178188047b6f48c"}},{"code_sha256_prefix":"f425d7b1bc9d8812","entry":"compute_gradients","repo":"elisanguyen/bayesian-tda","repo_kind":"official","path":"nn_influence_utils_vit.py","file_url":"https://github.com/elisanguyen/bayesian-tda/blob/HEAD/nn_influence_utils_vit.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f425d7b1bc9d8812"}},{"code_sha256_prefix":"c87c32b02c020f68","entry":"compute_hessian_vector_products","repo":"elisanguyen/bayesian-tda","repo_kind":"official","path":"nn_influence_utils_vit.py","file_url":"https://github.com/elisanguyen/bayesian-tda/blob/HEAD/nn_influence_utils_vit.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c87c32b02c020f68"}},{"code_sha256_prefix":"3cb0dc718b4eaf31","entry":"count_parameters","repo":"elisanguyen/bayesian-tda","repo_kind":"official","path":"nn_influence_utils.py","file_url":"https://github.com/elisanguyen/bayesian-tda/blob/HEAD/nn_influence_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3cb0dc718b4eaf31"}},{"code_sha256_prefix":"17f20ca8fa34ae6f","entry":"get_loss_with_weight_decay","repo":"elisanguyen/bayesian-tda","repo_kind":"official","path":"nn_influence_utils.py","file_url":"https://github.com/elisanguyen/bayesian-tda/blob/HEAD/nn_influence_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"17f20ca8fa34ae6f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}