{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unraveling-the-hessian-a-key-to-smooth","title":"Unraveling the Hessian: A Key to Smooth Convergence in Loss Function Landscapes","arxiv_id":"2409.11995","date":"2024-09-18","proceeding":null,"authors":["Nikita Kiselev","Andrey Grabovoy"],"abstract":"The loss landscape of neural networks is a critical aspect of their training, and understanding its properties is essential for improving their performance. In this paper, we investigate how the loss surface changes when the sample size increases, a previously unexplored issue. We theoretically analyze the convergence of the loss landscape in a fully connected neural network and derive upper bounds for the difference in loss function values when adding a new object to the sample. Our empirical study confirms these results on various datasets, demonstrating the convergence of the loss function surface for image classification tasks. Our findings provide insights into the local geometry of neural loss landscapes and have implications for the development of sample size determination techniques.","url_abs":"https://arxiv.org/abs/2409.11995v1","url_pdf":"https://arxiv.org/pdf/2409.11995v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unraveling-the-hessian-a-key-to-smooth","repo_url":"https://github.com/kisnikser/landscape-hessian","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2409.11995","atlas_url":"https://app.syntology.ai/?focus=2409.11995","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2409.11995"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/kisnikser/landscape-hessian","reach":null}],"summary":{"ran_draft_wrong":4},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"cf5d50a254b9b86e","entry":"calculate_ema","repo":"kisnikser/landscape-hessian","repo_kind":"official","path":"code/get_loss_values.py","file_url":"https://github.com/kisnikser/landscape-hessian/blob/HEAD/code/get_loss_values.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cf5d50a254b9b86e"}},{"code_sha256_prefix":"a7406e6dd4491589","entry":"calculate_ema","repo":"kisnikser/landscape-hessian","repo_kind":"official","path":"code/plot_differences.py","file_url":"https://github.com/kisnikser/landscape-hessian/blob/HEAD/code/plot_differences.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a7406e6dd4491589"}},{"code_sha256_prefix":"549ca336ff33b04d","entry":"criterion_params","repo":"kisnikser/landscape-hessian","repo_kind":"official","path":"code/get_loss_values.py","file_url":"https://github.com/kisnikser/landscape-hessian/blob/HEAD/code/get_loss_values.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"549ca336ff33b04d"}},{"code_sha256_prefix":"1c3c27dd0b929d96","entry":"get_loss_abs_differences","repo":"kisnikser/landscape-hessian","repo_kind":"official","path":"code/get_loss_values.py","file_url":"https://github.com/kisnikser/landscape-hessian/blob/HEAD/code/get_loss_values.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1c3c27dd0b929d96"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}