{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/robust-feature-learning-for-multi-index","title":"Robust Feature Learning for Multi-Index Models in High Dimensions","arxiv_id":"2410.16449","date":"2024-10-21","proceeding":null,"authors":["Alireza Mousavi-Hosseini","Adel Javanmard","Murat A. Erdogdu"],"abstract":"Recently, there have been numerous studies on feature learning with neural networks, specifically on learning single- and multi-index models where the target is a function of a low-dimensional projection of the input. Prior works have shown that in high dimensions, the majority of the compute and data resources are spent on recovering the low-dimensional projection; once this subspace is recovered, the remainder of the target can be learned independently of the ambient dimension. However, implications of feature learning in adversarial settings remain unexplored. In this work, we take the first steps towards understanding adversarially robust feature learning with neural networks. Specifically, we prove that the hidden directions of a multi-index model offer a Bayes optimal low-dimensional projection for robustness against $\\ell_2$-bounded adversarial perturbations under the squared loss, assuming that the multi-index coordinates are statistically independent from the rest of the coordinates. Therefore, robust learning can be achieved by first performing standard feature learning, then robustly tuning a linear readout layer on top of the standard representations. In particular, we show that adversarially robust learning is just as easy as standard learning. Specifically, the additional number of samples needed to robustly learn multi-index models when compared to standard learning does not depend on dimensionality.","url_abs":"https://arxiv.org/abs/2410.16449v2","url_pdf":"https://arxiv.org/pdf/2410.16449v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"robust-feature-learning-for-multi-index","repo_url":"https://github.com/mousavih/robust-feature-learning","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2410.16449","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2410.16449"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mousavih/robust-feature-learning","reach":null}],"summary":{"ran_draft_wrong":2,"unverified":4},"by_repo_kind":{"official":{"samples":6,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"75062a76fe17cdb7","entry":"cost","repo":"mousavih/robust-feature-learning","repo_kind":"official","path":"robust_fl.py","file_url":"https://github.com/mousavih/robust-feature-learning/blob/HEAD/robust_fl.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"75062a76fe17cdb7"}},{"code_sha256_prefix":"841afd9f88a39161","entry":"optimize_delta","repo":"mousavih/robust-feature-learning","repo_kind":"official","path":"robust_fl.py","file_url":"https://github.com/mousavih/robust-feature-learning/blob/HEAD/robust_fl.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"841afd9f88a39161"}},{"code_sha256_prefix":"51ce34e346e090da","entry":"He2","repo":"mousavih/robust-feature-learning","repo_kind":"official","path":"general_experiment.py","file_url":"https://github.com/mousavih/robust-feature-learning/blob/HEAD/general_experiment.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"51ce34e346e090da"}},{"code_sha256_prefix":"6222f818d24e085e","entry":"grad_params","repo":"mousavih/robust-feature-learning","repo_kind":"official","path":"robust_fl.py","file_url":"https://github.com/mousavih/robust-feature-learning/blob/HEAD/robust_fl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6222f818d24e085e"}},{"code_sha256_prefix":"fa0ba2ca0fbf09c6","entry":"run_experiment_known_dir","repo":"mousavih/robust-feature-learning","repo_kind":"official","path":"general_experiment.py","file_url":"https://github.com/mousavih/robust-feature-learning/blob/HEAD/general_experiment.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fa0ba2ca0fbf09c6"}},{"code_sha256_prefix":"0f9c6801b73bb211","entry":"run_experiment_unknown_dir","repo":"mousavih/robust-feature-learning","repo_kind":"official","path":"general_experiment.py","file_url":"https://github.com/mousavih/robust-feature-learning/blob/HEAD/general_experiment.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0f9c6801b73bb211"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}