{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/turning-the-tables-biased-imbalanced-dynamic","title":"Turning the Tables: Biased, Imbalanced, Dynamic Tabular Datasets for ML Evaluation","arxiv_id":"2211.13358","date":"2022-11-24","proceeding":null,"authors":["Sérgio Jesus","José Pombal","Duarte Alves","André Cruz","Pedro Saleiro","Rita P. Ribeiro","João Gama","Pedro Bizarro"],"abstract":"Evaluating new techniques on realistic datasets plays a crucial role in the development of ML research and its broader adoption by practitioners. In recent years, there has been a significant increase of publicly available unstructured data resources for computer vision and NLP tasks. However, tabular data -- which is prevalent in many high-stakes domains -- has been lagging behind. To bridge this gap, we present Bank Account Fraud (BAF), the first publicly available privacy-preserving, large-scale, realistic suite of tabular datasets. The suite was generated by applying state-of-the-art tabular data generation techniques on an anonymized,real-world bank account opening fraud detection dataset. This setting carries a set of challenges that are commonplace in real-world applications, including temporal dynamics and significant class imbalance. Additionally, to allow practitioners to stress test both performance and fairness of ML methods, each dataset variant of BAF contains specific types of data bias. With this resource, we aim to provide the research community with a more realistic, complete, and robust test bed to evaluate novel and existing methods.","url_abs":"https://arxiv.org/abs/2211.13358v2","url_pdf":"https://arxiv.org/pdf/2211.13358v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"turning-the-tables-biased-imbalanced-dynamic","repo_url":"https://github.com/feedzai/bank-account-fraud","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"turning-the-tables-biased-imbalanced-dynamic","repo_url":"https://github.com/dssg/aequitas","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"fairness","task_name":"Fairness"},{"task_slug":"fraud-detection","task_name":"Fraud Detection"},{"task_slug":"privacy-preserving","task_name":"Privacy Preserving"},{"task_slug":"tabular-data-generation","task_name":"Tabular Data Generation"}],"methods":[{"method_slug":"test","method_name":"Test"}],"datasets_introduced":[{"slug":"baf","name":"BAF","full_name":"Bank Account Fraud"}],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2211.13358","atlas_url":"https://app.syntology.ai/?focus=2211.13358","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2211.13358"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/feedzai/bank-account-fraud","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dssg/aequitas","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":6,"unverified":3},"by_repo_kind":{"listed":{"samples":9,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"732e3d1c4bfbe030","entry":"assemble_ref_groups","repo":"dssg/aequitas","repo_kind":"listed","path":"src/aequitas/plotting.py","file_url":"https://github.com/dssg/aequitas/blob/HEAD/src/aequitas/plotting.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"732e3d1c4bfbe030"}},{"code_sha256_prefix":"169ee466f281cd50","entry":"discretize","repo":"dssg/aequitas","repo_kind":"listed","path":"src/aequitas/preprocessing.py","file_url":"https://github.com/dssg/aequitas/blob/HEAD/src/aequitas/preprocessing.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"169ee466f281cd50"}},{"code_sha256_prefix":"312a590651b30772","entry":"get_attr_cols","repo":"dssg/aequitas","repo_kind":"listed","path":"src/aequitas/preprocessing.py","file_url":"https://github.com/dssg/aequitas/blob/HEAD/src/aequitas/preprocessing.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"312a590651b30772"}},{"code_sha256_prefix":"a54f09afe58b552d","entry":"layoutcol","repo":"dssg/aequitas","repo_kind":"listed","path":"src/aequitas/squarify_flipped.py","file_url":"https://github.com/dssg/aequitas/blob/HEAD/src/aequitas/squarify_flipped.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a54f09afe58b552d"}},{"code_sha256_prefix":"5ee63e07895636b5","entry":"layoutrow","repo":"dssg/aequitas","repo_kind":"listed","path":"src/aequitas/squarify_flipped.py","file_url":"https://github.com/dssg/aequitas/blob/HEAD/src/aequitas/squarify_flipped.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5ee63e07895636b5"}},{"code_sha256_prefix":"9a31b9f8b63762ae","entry":"normalize_sizes","repo":"dssg/aequitas","repo_kind":"listed","path":"src/aequitas/squarify_flipped.py","file_url":"https://github.com/dssg/aequitas/blob/HEAD/src/aequitas/squarify_flipped.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a31b9f8b63762ae"}},{"code_sha256_prefix":"cdb6611f9038db1f","entry":"audit","repo":"dssg/aequitas","repo_kind":"listed","path":"src/aequitas_cli/aequitas_audit.py","file_url":"https://github.com/dssg/aequitas/blob/HEAD/src/aequitas_cli/aequitas_audit.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cdb6611f9038db1f"}},{"code_sha256_prefix":"ea04a43eed8f8533","entry":"preprocess_input_df","repo":"dssg/aequitas","repo_kind":"listed","path":"src/aequitas/preprocessing.py","file_url":"https://github.com/dssg/aequitas/blob/HEAD/src/aequitas/preprocessing.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ea04a43eed8f8533"}},{"code_sha256_prefix":"3b4c242c8154d3e2","entry":"run","repo":"dssg/aequitas","repo_kind":"listed","path":"src/aequitas_cli/aequitas_audit.py","file_url":"https://github.com/dssg/aequitas/blob/HEAD/src/aequitas_cli/aequitas_audit.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3b4c242c8154d3e2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}