{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mechanism-of-feature-learning-in","title":"Mechanism of feature learning in convolutional neural networks","arxiv_id":"2309.00570","date":"2023-09-01","proceeding":null,"authors":["Daniel Beaglehole","Adityanarayanan Radhakrishnan","Parthe Pandit","Mikhail Belkin"],"abstract":"Understanding the mechanism of how convolutional neural networks learn features from image data is a fundamental problem in machine learning and computer vision. In this work, we identify such a mechanism. We posit the Convolutional Neural Feature Ansatz, which states that covariances of filters in any convolutional layer are proportional to the average gradient outer product (AGOP) taken with respect to patches of the input to that layer. We present extensive empirical evidence for our ansatz, including identifying high correlation between covariances of filters and patch-based AGOPs for convolutional layers in standard neural architectures, such as AlexNet, VGG, and ResNets pre-trained on ImageNet. We also provide supporting theoretical evidence. We then demonstrate the generality of our result by using the patch-based AGOP to enable deep feature learning in convolutional kernel machines. We refer to the resulting algorithm as (Deep) ConvRFM and show that our algorithm recovers similar features to deep convolutional networks including the notable emergence of edge detectors. Moreover, we find that Deep ConvRFM overcomes previously identified limitations of convolutional kernels, such as their inability to adapt to local signals in images and, as a result, leads to sizable performance improvement over fixed convolutional kernels.","url_abs":"https://arxiv.org/abs/2309.00570v1","url_pdf":"https://arxiv.org/pdf/2309.00570v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mechanism-of-feature-learning-in","repo_url":"https://github.com/aradha/convrfm","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"jax","reach":{"status":"ok"}}],"tasks":[],"methods":[{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2309.00570","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.00570"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/aradha/convrfm","reach":{"status":"ok"}}],"summary":{"ran":4,"unverified":13},"by_repo_kind":{"official":{"samples":17,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":17,"samples":[{"code_sha256_prefix":"b0e4baca69a85d2c","entry":"ConvMixer","repo":"aradha/convrfm","repo_kind":"official","path":"conv_nets/models.py","file_url":"https://github.com/aradha/convrfm/blob/HEAD/conv_nets/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b0e4baca69a85d2c"}},{"code_sha256_prefix":"7caf93c86037eccb","entry":"get_binary","repo":"aradha/convrfm","repo_kind":"official","path":"conv_nets/dataset.py","file_url":"https://github.com/aradha/convrfm/blob/HEAD/conv_nets/dataset.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7caf93c86037eccb"}},{"code_sha256_prefix":"84911c24475ad37c","entry":"get_filter","repo":"aradha/convrfm","repo_kind":"official","path":"conv_nets/binary_main.py","file_url":"https://github.com/aradha/convrfm/blob/HEAD/conv_nets/binary_main.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"84911c24475ad37c"}},{"code_sha256_prefix":"a9233c8c95aa8eb6","entry":"get_param_count","repo":"aradha/convrfm","repo_kind":"official","path":"conv_nets/models.py","file_url":"https://github.com/aradha/convrfm/blob/HEAD/conv_nets/models.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a9233c8c95aa8eb6"}},{"code_sha256_prefix":"64ac5d19d1771578","entry":"egop","repo":"aradha/convrfm","repo_kind":"official","path":"cnfa_verification/pretrained_conv_nfa.py","file_url":"https://github.com/aradha/convrfm/blob/HEAD/cnfa_verification/pretrained_conv_nfa.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"64ac5d19d1771578"}},{"code_sha256_prefix":"f522f27f5938d086","entry":"gen_toy_mnist_backgrounds","repo":"aradha/convrfm","repo_kind":"official","path":"gen_toy_data.py","file_url":"https://github.com/aradha/convrfm/blob/HEAD/gen_toy_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f522f27f5938d086"}},{"code_sha256_prefix":"00500ae5d0cf1a9e","entry":"get_cats_dogs","repo":"aradha/convrfm","repo_kind":"official","path":"conv_nets/utils.py","file_url":"https://github.com/aradha/convrfm/blob/HEAD/conv_nets/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"00500ae5d0cf1a9e"}},{"code_sha256_prefix":"f9033f488349f9e8","entry":"get_classes","repo":"aradha/convrfm","repo_kind":"official","path":"conv_nets/binary_main.py","file_url":"https://github.com/aradha/convrfm/blob/HEAD/conv_nets/binary_main.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f9033f488349f9e8"}},{"code_sha256_prefix":"4f0e5a32e15f56a8","entry":"get_jacobian","repo":"aradha/convrfm","repo_kind":"official","path":"cnfa_verification/pretrained_conv_nfa.py","file_url":"https://github.com/aradha/convrfm/blob/HEAD/cnfa_verification/pretrained_conv_nfa.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"4f0e5a32e15f56a8"}},{"code_sha256_prefix":"87ff0adfd10b9493","entry":"get_toy","repo":"aradha/convrfm","repo_kind":"official","path":"conv_nets/dataset.py","file_url":"https://github.com/aradha/convrfm/blob/HEAD/conv_nets/dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"87ff0adfd10b9493"}},{"code_sha256_prefix":"feadac03d2798a03","entry":"matrix_sqrt","repo":"aradha/convrfm","repo_kind":"official","path":"conv_nets/utils.py","file_url":"https://github.com/aradha/convrfm/blob/HEAD/conv_nets/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"feadac03d2798a03"}},{"code_sha256_prefix":"1414fbc4731c4121","entry":"my_svd","repo":"aradha/convrfm","repo_kind":"official","path":"conv_nets/utils.py","file_url":"https://github.com/aradha/convrfm/blob/HEAD/conv_nets/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1414fbc4731c4121"}},{"code_sha256_prefix":"18eda8c222c24cf7","entry":"one_hot_data","repo":"aradha/convrfm","repo_kind":"official","path":"conv_nets/dataset.py","file_url":"https://github.com/aradha/convrfm/blob/HEAD/conv_nets/dataset.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"18eda8c222c24cf7"}},{"code_sha256_prefix":"e0c0dc70dcaf23d7","entry":"patchify","repo":"aradha/convrfm","repo_kind":"official","path":"cnfa_verification/pretrained_conv_nfa.py","file_url":"https://github.com/aradha/convrfm/blob/HEAD/cnfa_verification/pretrained_conv_nfa.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e0c0dc70dcaf23d7"}},{"code_sha256_prefix":"0cf3b5fb0cd3cf4e","entry":"train_network","repo":"aradha/convrfm","repo_kind":"official","path":"conv_nets/trainer.py","file_url":"https://github.com/aradha/convrfm/blob/HEAD/conv_nets/trainer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0cf3b5fb0cd3cf4e"}},{"code_sha256_prefix":"8742a8050792c3dd","entry":"train_step","repo":"aradha/convrfm","repo_kind":"official","path":"conv_nets/trainer.py","file_url":"https://github.com/aradha/convrfm/blob/HEAD/conv_nets/trainer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8742a8050792c3dd"}},{"code_sha256_prefix":"c53a2f9b5dd8e419","entry":"val_step","repo":"aradha/convrfm","repo_kind":"official","path":"conv_nets/trainer.py","file_url":"https://github.com/aradha/convrfm/blob/HEAD/conv_nets/trainer.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c53a2f9b5dd8e419"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}