{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/implicit-bias-of-linear-equivariant-networks-1","title":"Implicit Bias of Linear Equivariant Networks","arxiv_id":"2110.06084","date":"2021-10-12","proceeding":null,"authors":["Hannah Lawrence","Kristian Georgiev","Andrew Dienes","Bobak T. Kiani"],"abstract":"Group equivariant convolutional neural networks (G-CNNs) are generalizations of convolutional neural networks (CNNs) which excel in a wide range of technical applications by explicitly encoding symmetries, such as rotations and permutations, in their architectures. Although the success of G-CNNs is driven by their \\emph{explicit} symmetry bias, a recent line of work has proposed that the \\emph{implicit} bias of training algorithms on particular architectures is key to understanding generalization for overparameterized neural nets. In this context, we show that $L$-layer full-width linear G-CNNs trained via gradient descent for binary classification converge to solutions with low-rank Fourier matrix coefficients, regularized by the $2/L$-Schatten matrix norm. Our work strictly generalizes previous analysis on the implicit bias of linear CNNs to linear G-CNNs over all finite groups, including the challenging setting of non-commutative groups (such as permutations), as well as band-limited G-CNNs over infinite groups. We validate our theorems via experiments on a variety of groups, and empirically explore more realistic nonlinear networks, which locally capture similar regularization patterns. Finally, we provide intuitive interpretations of our Fourier space implicit regularization results in real space via uncertainty principles.","url_abs":"https://arxiv.org/abs/2110.06084v3","url_pdf":"https://arxiv.org/pdf/2110.06084v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"implicit-bias-of-linear-equivariant-networks-1","repo_url":"https://github.com/kristian-georgiev/implicit-bias-of-linear-equivariant-networks","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"binary-classification","task_name":"Binary Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2110.06084","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2110.06084"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kristian-georgiev/implicit-bias-of-linear-equivariant-networks","reach":null}],"summary":{"ran_violates":1,"ran_draft_wrong":1,"ran_fixture":1},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5800bb7082be8718","entry":"exp_loss","repo":"kristian-georgiev/implicit-bias-of-linear-equivariant-networks","repo_kind":"official","path":"src/nn_training.py","file_url":"https://github.com/kristian-georgiev/implicit-bias-of-linear-equivariant-networks/blob/HEAD/src/nn_training.py","link_basis":"first_harvest_node","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5800bb7082be8718"}},{"code_sha256_prefix":"3de500e47216d151","entry":"g_conv_func","repo":"kristian-georgiev/implicit-bias-of-linear-equivariant-networks","repo_kind":"official","path":"src/nn_training.py","file_url":"https://github.com/kristian-georgiev/implicit-bias-of-linear-equivariant-networks/blob/HEAD/src/nn_training.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3de500e47216d151"}},{"code_sha256_prefix":"3bb3c6c08de35284","entry":"permutation_matrix","repo":"kristian-georgiev/implicit-bias-of-linear-equivariant-networks","repo_kind":"official","path":"src/nn_training.py","file_url":"https://github.com/kristian-georgiev/implicit-bias-of-linear-equivariant-networks/blob/HEAD/src/nn_training.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3bb3c6c08de35284"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}