{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mabsplit-faster-forest-training-using-multi","title":"MABSplit: Faster Forest Training Using Multi-Armed Bandits","arxiv_id":"2212.07473","date":"2022-12-14","proceeding":null,"authors":["Mo Tiwari","Ryan Kang","Je-Yong Lee","Sebastian Thrun","Chris Piech","Ilan Shomorony","Martin Jinye Zhang"],"abstract":"Random forests are some of the most widely used machine learning models today, especially in domains that necessitate interpretability. We present an algorithm that accelerates the training of random forests and other popular tree-based learning methods. At the core of our algorithm is a novel node-splitting subroutine, dubbed MABSplit, used to efficiently find split points when constructing decision trees. Our algorithm borrows techniques from the multi-armed bandit literature to judiciously determine how to allocate samples and computational power across candidate split points. We provide theoretical guarantees that MABSplit improves the sample complexity of each node split from linear to logarithmic in the number of data points. In some settings, MABSplit leads to 100x faster training (an 99% reduction in training time) without any decrease in generalization performance. We demonstrate similar speedups when MABSplit is used across a variety of forest-based variants, such as Extremely Random Forests and Random Patches. We also show our algorithm can be used in both classification and regression tasks. Finally, we show that MABSplit outperforms existing methods in generalization performance and feature importance calculations under a fixed computational budget. All of our experimental results are reproducible via a one-line script at https://github.com/ThrunGroup/FastForest.","url_abs":"https://arxiv.org/abs/2212.07473v1","url_pdf":"https://arxiv.org/pdf/2212.07473v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mabsplit-faster-forest-training-using-multi","repo_url":"https://github.com/thrungroup/fastforest","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"feature-importance","task_name":"Feature Importance"},{"task_slug":"multi-armed-bandits","task_name":"Multi-Armed Bandits"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2212.07473","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2212.07473"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/ThrunGroup/FastForest","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/thrungroup/fastforest","reach":{"status":"ok"}}],"summary":{"ran":1,"ran_draft_wrong":2,"ran_fixture":3,"ran_honours":1,"unverified":7},"by_repo_kind":{"official":{"samples":14,"ran":7,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":14,"samples":[{"code_sha256_prefix":"186f8aa9fefbb971","entry":"Histogram","repo":"thrungroup/fastforest","repo_kind":"official","path":"utils/solvers.py","file_url":"https://github.com/thrungroup/fastforest/blob/HEAD/utils/solvers.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"186f8aa9fefbb971"}},{"code_sha256_prefix":"0753bfbc13b72385","entry":"choose_bin_type","repo":"thrungroup/fastforest","repo_kind":"official","path":"utils/solvers.py","file_url":"https://github.com/thrungroup/fastforest/blob/HEAD/utils/solvers.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0753bfbc13b72385"}},{"code_sha256_prefix":"8a5b76b92538164a","entry":"class_to_idx","repo":"thrungroup/fastforest","repo_kind":"official","path":"utils/solvers.py","file_url":"https://github.com/thrungroup/fastforest/blob/HEAD/utils/solvers.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8a5b76b92538164a"}},{"code_sha256_prefix":"c11ef2293c1894fc","entry":"get_entropy","repo":"thrungroup/fastforest","repo_kind":"official","path":"utils/solvers.py","file_url":"https://github.com/thrungroup/fastforest/blob/HEAD/utils/solvers.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c11ef2293c1894fc"}},{"code_sha256_prefix":"bdb81f9073baf931","entry":"get_gini","repo":"thrungroup/fastforest","repo_kind":"official","path":"utils/solvers.py","file_url":"https://github.com/thrungroup/fastforest/blob/HEAD/utils/solvers.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bdb81f9073baf931"}},{"code_sha256_prefix":"2ccb8b761652b05e","entry":"get_mse","repo":"thrungroup/fastforest","repo_kind":"official","path":"utils/solvers.py","file_url":"https://github.com/thrungroup/fastforest/blob/HEAD/utils/solvers.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2ccb8b761652b05e"}},{"code_sha256_prefix":"142b3ed4bdb7655f","entry":"welford_variance_calc","repo":"thrungroup/fastforest","repo_kind":"official","path":"utils/solvers.py","file_url":"https://github.com/thrungroup/fastforest/blob/HEAD/utils/solvers.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"142b3ed4bdb7655f"}},{"code_sha256_prefix":"710b4b08c61beede","entry":"empty_histograms","repo":"thrungroup/fastforest","repo_kind":"official","path":"utils/solvers.py","file_url":"https://github.com/thrungroup/fastforest/blob/HEAD/utils/solvers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"710b4b08c61beede"}},{"code_sha256_prefix":"58d5bd11c8ad825b","entry":"get_impurity_fn","repo":"thrungroup/fastforest","repo_kind":"official","path":"utils/solvers.py","file_url":"https://github.com/thrungroup/fastforest/blob/HEAD/utils/solvers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"58d5bd11c8ad825b"}},{"code_sha256_prefix":"c67e06269033ed88","entry":"get_impurity_reductions","repo":"thrungroup/fastforest","repo_kind":"official","path":"utils/solvers.py","file_url":"https://github.com/thrungroup/fastforest/blob/HEAD/utils/solvers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c67e06269033ed88"}},{"code_sha256_prefix":"06f76a1bb6683cb8","entry":"make_histograms","repo":"thrungroup/fastforest","repo_kind":"official","path":"utils/solvers.py","file_url":"https://github.com/thrungroup/fastforest/blob/HEAD/utils/solvers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"06f76a1bb6683cb8"}},{"code_sha256_prefix":"d7020c0c5dc36ace","entry":"sample_targets","repo":"thrungroup/fastforest","repo_kind":"official","path":"utils/solvers.py","file_url":"https://github.com/thrungroup/fastforest/blob/HEAD/utils/solvers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d7020c0c5dc36ace"}},{"code_sha256_prefix":"f94e0a35fecdcb2b","entry":"solve_mab","repo":"thrungroup/fastforest","repo_kind":"official","path":"utils/solvers.py","file_url":"https://github.com/thrungroup/fastforest/blob/HEAD/utils/solvers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f94e0a35fecdcb2b"}},{"code_sha256_prefix":"d9ea6ab91cff145f","entry":"time_measured_fit","repo":"thrungroup/fastforest","repo_kind":"official","path":"experiments/runtime_exps/compare_runtimes.py","file_url":"https://github.com/thrungroup/fastforest/blob/HEAD/experiments/runtime_exps/compare_runtimes.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d9ea6ab91cff145f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}