{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/random-feature-representation-boosting","title":"Random Feature Representation Boosting","arxiv_id":"2501.18283","date":"2025-01-30","proceeding":null,"authors":["Nikita Zozoulenko","Thomas Cass","Lukas Gonon"],"abstract":"We introduce Random Feature Representation Boosting (RFRBoost), a novel method for constructing deep residual random feature neural networks (RFNNs) using boosting theory. RFRBoost uses random features at each layer to learn the functional gradient of the network representation, enhancing performance while preserving the convex optimization benefits of RFNNs. In the case of MSE loss, we obtain closed-form solutions to greedy layer-wise boosting with random features. For general loss functions, we show that fitting random feature residual blocks reduces to solving a quadratically constrained least squares problem. We demonstrate, through numerical experiments on 91 tabular datasets for regression and classification, that RFRBoost significantly outperforms traditional RFNNs and end-to-end trained MLP ResNets, while offering substantial computational advantages and theoretical guarantees stemming from boosting theory.","url_abs":"https://arxiv.org/abs/2501.18283v1","url_pdf":"https://arxiv.org/pdf/2501.18283v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"random-feature-representation-boosting","repo_url":"https://github.com/nikitazozoulenko/random-feature-representation-boosting","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2501.18283","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2501.18283"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nikitazozoulenko/random-feature-representation-boosting","reach":null}],"summary":{"ran":3,"ran_draft_wrong":2,"unverified":4},"by_repo_kind":{"official":{"samples":9,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"f13f157aeaf67ba5","entry":"FittableModule","repo":"nikitazozoulenko/random-feature-representation-boosting","repo_kind":"official","path":"models/random_feature_representation_boosting.py","file_url":"https://github.com/nikitazozoulenko/random-feature-representation-boosting/blob/HEAD/models/random_feature_representation_boosting.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f13f157aeaf67ba5"}},{"code_sha256_prefix":"9452a17746aa4396","entry":"SWIMLayer","repo":"nikitazozoulenko/random-feature-representation-boosting","repo_kind":"official","path":"models/random_feature_representation_boosting.py","file_url":"https://github.com/nikitazozoulenko/random-feature-representation-boosting/blob/HEAD/models/random_feature_representation_boosting.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9452a17746aa4396"}},{"code_sha256_prefix":"c2d5f18425eeec9d","entry":"ScaleLayer","repo":"nikitazozoulenko/random-feature-representation-boosting","repo_kind":"official","path":"models/random_feature_representation_boosting.py","file_url":"https://github.com/nikitazozoulenko/random-feature-representation-boosting/blob/HEAD/models/random_feature_representation_boosting.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c2d5f18425eeec9d"}},{"code_sha256_prefix":"7ca2a6661b66eaec","entry":"create_layer","repo":"nikitazozoulenko/random-feature-representation-boosting","repo_kind":"official","path":"models/random_feature_representation_boosting.py","file_url":"https://github.com/nikitazozoulenko/random-feature-representation-boosting/blob/HEAD/models/random_feature_representation_boosting.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7ca2a6661b66eaec"}},{"code_sha256_prefix":"a03ddcfdcf0c0868","entry":"make_fittable","repo":"nikitazozoulenko/random-feature-representation-boosting","repo_kind":"official","path":"models/random_feature_representation_boosting.py","file_url":"https://github.com/nikitazozoulenko/random-feature-representation-boosting/blob/HEAD/models/random_feature_representation_boosting.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a03ddcfdcf0c0868"}},{"code_sha256_prefix":"6c5456cff2e2a5fe","entry":"BaseGRFRBoost","repo":"nikitazozoulenko/random-feature-representation-boosting","repo_kind":"official","path":"models/random_feature_representation_boosting.py","file_url":"https://github.com/nikitazozoulenko/random-feature-representation-boosting/blob/HEAD/models/random_feature_representation_boosting.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6c5456cff2e2a5fe"}},{"code_sha256_prefix":"17fb7c40d07380a6","entry":"GhatBoostingLayer","repo":"nikitazozoulenko/random-feature-representation-boosting","repo_kind":"official","path":"models/random_feature_representation_boosting.py","file_url":"https://github.com/nikitazozoulenko/random-feature-representation-boosting/blob/HEAD/models/random_feature_representation_boosting.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"17fb7c40d07380a6"}},{"code_sha256_prefix":"e5407f0182a3f60b","entry":"RandomFeatureLayer","repo":"nikitazozoulenko/random-feature-representation-boosting","repo_kind":"official","path":"models/random_feature_representation_boosting.py","file_url":"https://github.com/nikitazozoulenko/random-feature-representation-boosting/blob/HEAD/models/random_feature_representation_boosting.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e5407f0182a3f60b"}},{"code_sha256_prefix":"a83fde14f6ef9d33","entry":"Upscale","repo":"nikitazozoulenko/random-feature-representation-boosting","repo_kind":"official","path":"models/random_feature_representation_boosting.py","file_url":"https://github.com/nikitazozoulenko/random-feature-representation-boosting/blob/HEAD/models/random_feature_representation_boosting.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a83fde14f6ef9d33"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}