{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/binning-as-a-pretext-task-improving-self","title":"Binning as a Pretext Task: Improving Self-Supervised Learning in Tabular Domains","arxiv_id":"2405.07414","date":"2024-05-13","proceeding":null,"authors":["Kyungeun Lee","Ye Seul Sim","Hye-Seung Cho","Moonjung Eo","Suhee Yoon","Sanghyu Yoon","Woohyung Lim"],"abstract":"The ability of deep networks to learn superior representations hinges on leveraging the proper inductive biases, considering the inherent properties of datasets. In tabular domains, it is critical to effectively handle heterogeneous features (both categorical and numerical) in a unified manner and to grasp irregular functions like piecewise constant functions. To address the challenges in the self-supervised learning framework, we propose a novel pretext task based on the classical binning method. The idea is straightforward: reconstructing the bin indices (either orders or classes) rather than the original values. This pretext task provides the encoder with an inductive bias to capture the irregular dependencies, mapping from continuous inputs to discretized bins, and mitigates the feature heterogeneity by setting all features to have category-type targets. Our empirical investigations ascertain several advantages of binning: capturing the irregular function, compatibility with encoder architecture and additional modifications, standardizing all features into equal sets, grouping similar values within a feature, and providing ordering information. Comprehensive evaluations across diverse tabular datasets corroborate that our method consistently improves tabular representation learning performance for a wide range of downstream tasks. The codes are available in https://github.com/kyungeun-lee/tabularbinning.","url_abs":"https://arxiv.org/abs/2405.07414v2","url_pdf":"https://arxiv.org/pdf/2405.07414v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"binning-as-a-pretext-task-improving-self","repo_url":"https://github.com/kyungeun-lee/tabularbinning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"binning-as-a-pretext-task-improving-self","repo_url":"https://github.com/Alcoholrithm/TabularS3L","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"inductive-bias","task_name":"Inductive Bias"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2405.07414","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.07414"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/kyungeun-lee/tabularbinning","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Alcoholrithm/TabularS3L","reach":null}],"summary":{"ran":4,"unverified":2},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1},"listed":{"samples":3,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"97482010d1e20926","entry":"CosineAnnealingParam","repo":"kyungeun-lee/tabularbinning","repo_kind":"official","path":"libs/model.py","file_url":"https://github.com/kyungeun-lee/tabularbinning/blob/HEAD/libs/model.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"97482010d1e20926"}},{"code_sha256_prefix":"0d5fe617b6ac8aa1","entry":"MLP","repo":"Alcoholrithm/TabularS3L","repo_kind":"listed","path":"ts3l/models/tabularbinning/tabularbinning.py","file_url":"https://github.com/Alcoholrithm/TabularS3L/blob/HEAD/ts3l/models/tabularbinning/tabularbinning.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0d5fe617b6ac8aa1"}},{"code_sha256_prefix":"041c3016c2ea1be3","entry":"get_batch_size","repo":"kyungeun-lee/tabularbinning","repo_kind":"official","path":"libs/utils.py","file_url":"https://github.com/kyungeun-lee/tabularbinning/blob/HEAD/libs/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"041c3016c2ea1be3"}},{"code_sha256_prefix":"2d0ed7a83a7870e1","entry":"replace_cat_to_num","repo":"kyungeun-lee/tabularbinning","repo_kind":"official","path":"libs/utils.py","file_url":"https://github.com/kyungeun-lee/tabularbinning/blob/HEAD/libs/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2d0ed7a83a7870e1"}},{"code_sha256_prefix":"effc94861e846a7b","entry":"TS3LModule","repo":"Alcoholrithm/TabularS3L","repo_kind":"listed","path":"ts3l/models/tabularbinning/tabularbinning.py","file_url":"https://github.com/Alcoholrithm/TabularS3L/blob/HEAD/ts3l/models/tabularbinning/tabularbinning.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"effc94861e846a7b"}},{"code_sha256_prefix":"45ff981ce249b013","entry":"TabularBinning","repo":"Alcoholrithm/TabularS3L","repo_kind":"listed","path":"ts3l/models/tabularbinning/tabularbinning.py","file_url":"https://github.com/Alcoholrithm/TabularS3L/blob/HEAD/ts3l/models/tabularbinning/tabularbinning.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"45ff981ce249b013"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}