{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-unified-framework-for-dataset-shift","title":"A unified framework for dataset shift diagnostics","arxiv_id":"2205.08340","date":"2022-05-17","proceeding":null,"authors":["Felipe Maia Polo","Rafael Izbicki","Evanildo Gomes Lacerda Jr","Juan Pablo Ibieta-Jimenez","Renato Vicente"],"abstract":"Supervised learning techniques typically assume training data originates from the target population. Yet, in reality, dataset shift frequently arises, which, if not adequately taken into account, may decrease the performance of their predictors. In this work, we propose a novel and flexible framework called DetectShift that quantifies and tests for multiple dataset shifts, encompassing shifts in the distributions of $(X, Y)$, $X$, $Y$, $X|Y$, and $Y|X$. DetectShift equips practitioners with insights into data shifts, facilitating the adaptation or retraining of predictors using both source and target data. This proves extremely valuable when labeled samples in the target domain are limited. The framework utilizes test statistics with the same nature to quantify the magnitude of the various shifts, making results more interpretable. It is versatile, suitable for regression and classification tasks, and accommodates diverse data forms - tabular, text, or image. Experimental results demonstrate the effectiveness of DetectShift in detecting dataset shifts even in higher dimensions.","url_abs":"https://arxiv.org/abs/2205.08340v4","url_pdf":"https://arxiv.org/pdf/2205.08340v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-unified-framework-for-dataset-shift","repo_url":"https://github.com/felipemaiapolo/dataset_shift_diagnostics","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"a-unified-framework-for-dataset-shift","repo_url":"https://github.com/felipemaiapolo/detectshift","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2205.08340","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2205.08340"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/felipemaiapolo/detectshift","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/felipemaiapolo/dataset_shift_diagnostics","reach":{"status":"ok"}}],"summary":{"ran_violates":1,"ran_honours":1,"ran_fixture":1},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"de8dbae9e4ef5bad","entry":"get_classes","repo":"felipemaiapolo/detectshift","repo_kind":"official","path":"detectshift/tools.py","file_url":"https://github.com/felipemaiapolo/detectshift/blob/HEAD/detectshift/tools.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"de8dbae9e4ef5bad"}},{"code_sha256_prefix":"2486610e2e422ed0","entry":"get_dummies","repo":"felipemaiapolo/detectshift","repo_kind":"official","path":"detectshift/tools.py","file_url":"https://github.com/felipemaiapolo/detectshift/blob/HEAD/detectshift/tools.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2486610e2e422ed0"}},{"code_sha256_prefix":"21597cb5fc966cd1","entry":"prep_data","repo":"felipemaiapolo/detectshift","repo_kind":"official","path":"detectshift/tools.py","file_url":"https://github.com/felipemaiapolo/detectshift/blob/HEAD/detectshift/tools.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"21597cb5fc966cd1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}