{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/lazy-vs-hasty-linearization-in-deep-networks","title":"Lazy vs hasty: linearization in deep networks impacts learning schedule based on example difficulty","arxiv_id":"2209.09658","date":"2022-09-19","proceeding":null,"authors":["Thomas George","Guillaume Lajoie","Aristide Baratin"],"abstract":"Among attempts at giving a theoretical account of the success of deep neural networks, a recent line of work has identified a so-called lazy training regime in which the network can be well approximated by its linearization around initialization. Here we investigate the comparative effect of the lazy (linear) and feature learning (non-linear) regimes on subgroups of examples based on their difficulty. Specifically, we show that easier examples are given more weight in feature learning mode, resulting in faster training compared to more difficult ones. In other words, the non-linear dynamics tends to sequentialize the learning of examples of increasing difficulty. We illustrate this phenomenon across different ways to quantify example difficulty, including c-score, label noise, and in the presence of easy-to-learn spurious correlations. Our results reveal a new understanding of how deep networks prioritize resources across example difficulty.","url_abs":"https://arxiv.org/abs/2209.09658v2","url_pdf":"https://arxiv.org/pdf/2209.09658v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"lazy-vs-hasty-linearization-in-deep-networks","repo_url":"https://github.com/tfjgeorge/lazy_vs_hasty","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2209.09658","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2209.09658"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/tfjgeorge/lazy_vs_hasty","reach":null}],"summary":{"ran_draft_wrong":2,"unverified":1},"by_repo_kind":{"official":{"samples":3,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"ed58afd3ae5f16d6","entry":"conv1x1","repo":"tfjgeorge/lazy_vs_hasty","repo_kind":"official","path":"exp_classif/models.py","file_url":"https://github.com/tfjgeorge/lazy_vs_hasty/blob/HEAD/exp_classif/models.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ed58afd3ae5f16d6"}},{"code_sha256_prefix":"15773279a9becba1","entry":"conv3x3","repo":"tfjgeorge/lazy_vs_hasty","repo_kind":"official","path":"exp_classif/models.py","file_url":"https://github.com/tfjgeorge/lazy_vs_hasty/blob/HEAD/exp_classif/models.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"15773279a9becba1"}},{"code_sha256_prefix":"8061bb1404e896ff","entry":"resnet18","repo":"tfjgeorge/lazy_vs_hasty","repo_kind":"official","path":"exp_classif/models.py","file_url":"https://github.com/tfjgeorge/lazy_vs_hasty/blob/HEAD/exp_classif/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8061bb1404e896ff"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}