{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-a-decision-tree-algorithm-with","title":"Learning a Decision Tree Algorithm with Transformers","arxiv_id":"2402.03774","date":"2024-02-06","proceeding":null,"authors":["Yufan Zhuang","Liyuan Liu","Chandan Singh","Jingbo Shang","Jianfeng Gao"],"abstract":"Decision trees are renowned for their ability to achieve high predictive performance while remaining interpretable, especially on tabular data. Traditionally, they are constructed through recursive algorithms, where they partition the data at every node in a tree. However, identifying a good partition is challenging, as decision trees optimized for local segments may not yield global generalization. To address this, we introduce MetaTree, a transformer-based model trained via meta-learning to directly produce strong decision trees. Specifically, we fit both greedy decision trees and globally optimized decision trees on a large number of datasets, and train MetaTree to produce only the trees that achieve strong generalization performance. This training enables MetaTree to emulate these algorithms and intelligently adapt its strategy according to the context, thereby achieving superior generalization performance.","url_abs":"https://arxiv.org/abs/2402.03774v2","url_pdf":"https://arxiv.org/pdf/2402.03774v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-a-decision-tree-algorithm-with","repo_url":"https://github.com/evanzhuang/metatree","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"meta-learning","task_name":"Meta-Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2402.03774","atlas_url":"https://app.syntology.ai/?focus=2402.03774","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.03774"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/evanzhuang/metatree","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":5,"ran_draft_wrong":1,"unverified":2},"by_repo_kind":{"official":{"samples":8,"ran":6,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"aaed837329b05922","entry":"entropy","repo":"evanzhuang/metatree","repo_kind":"official","path":"metatree/model_metatree.py","file_url":"https://github.com/evanzhuang/metatree/blob/HEAD/metatree/model_metatree.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aaed837329b05922"}},{"code_sha256_prefix":"7b71149c0d3a3d6d","entry":"entropy_construction","repo":"evanzhuang/metatree","repo_kind":"official","path":"metatree/model_metatree.py","file_url":"https://github.com/evanzhuang/metatree/blob/HEAD/metatree/model_metatree.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7b71149c0d3a3d6d"}},{"code_sha256_prefix":"aa7ad3fc695f98b5","entry":"get_restarting_cosine_schedule_with_warmup","repo":"evanzhuang/metatree","repo_kind":"official","path":"metatree/utils.py","file_url":"https://github.com/evanzhuang/metatree/blob/HEAD/metatree/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa7ad3fc695f98b5"}},{"code_sha256_prefix":"9753211d46e1379a","entry":"preprocess_dimension_patch","repo":"evanzhuang/metatree","repo_kind":"official","path":"metatree/run_train.py","file_url":"https://github.com/evanzhuang/metatree/blob/HEAD/metatree/run_train.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9753211d46e1379a"}},{"code_sha256_prefix":"b99eea6376d1e212","entry":"rotate_half","repo":"evanzhuang/metatree","repo_kind":"official","path":"metatree/model_metatree.py","file_url":"https://github.com/evanzhuang/metatree/blob/HEAD/metatree/model_metatree.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b99eea6376d1e212"}},{"code_sha256_prefix":"a74cb17b7e359391","entry":"tree_eval_process","repo":"evanzhuang/metatree","repo_kind":"official","path":"metatree/eval_generalization.py","file_url":"https://github.com/evanzhuang/metatree/blob/HEAD/metatree/eval_generalization.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a74cb17b7e359391"}},{"code_sha256_prefix":"a9c14b9fb71dbe7c","entry":"get_cosine_schedule_with_warmup","repo":"evanzhuang/metatree","repo_kind":"official","path":"metatree/utils.py","file_url":"https://github.com/evanzhuang/metatree/blob/HEAD/metatree/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a9c14b9fb71dbe7c"}},{"code_sha256_prefix":"afba2c06a42074fe","entry":"get_linear_schedule_with_warmup","repo":"evanzhuang/metatree","repo_kind":"official","path":"metatree/utils.py","file_url":"https://github.com/evanzhuang/metatree/blob/HEAD/metatree/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"afba2c06a42074fe"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}