{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-learning-theory-review-an-optimal","title":"Deep Learning Theory Review: An Optimal Control and Dynamical Systems Perspective","arxiv_id":"1908.10920","date":"2019-08-28","proceeding":null,"authors":["Guan-Horng Liu","Evangelos A. Theodorou"],"abstract":"Attempts from different disciplines to provide a fundamental understanding of deep learning have advanced rapidly in recent years, yet a unified framework remains relatively limited. In this article, we provide one possible way to align existing branches of deep learning theory through the lens of dynamical system and optimal control. By viewing deep neural networks as discrete-time nonlinear dynamical systems, we can analyze how information propagates through layers using mean field theory. When optimization algorithms are further recast as controllers, the ultimate goal of training processes can be formulated as an optimal control problem. In addition, we can reveal convergence and generalization properties by studying the stochastic dynamics of optimization algorithms. This viewpoint features a wide range of theoretical study from information bottleneck to statistical physics. It also provides a principled way for hyper-parameter tuning when optimal control theory is introduced. Our framework fits nicely with supervised learning and can be extended to other learning problems, such as Bayesian learning, adversarial training, and specific forms of meta learning, without efforts. The review aims to shed lights on the importance of dynamics and optimal control when developing deep learning theory.","url_abs":"https://arxiv.org/abs/1908.10920v2","url_pdf":"https://arxiv.org/pdf/1908.10920v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-learning-theory-review-an-optimal","repo_url":"https://github.com/ghliu/mean-field-fcdnn","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"learning-theory","task_name":"Learning Theory"},{"task_slug":"meta-learning","task_name":"Meta-Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1908.10920","atlas_url":"https://app.syntology.ai/?focus=1908.10920","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1908.10920"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ghliu/mean-field-fcdnn","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":5},"by_repo_kind":{"official":{"samples":5,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9f86e91daed794c9","entry":"compute_training_acc","repo":"ghliu/mean-field-fcdnn","repo_kind":"official","path":"train_acc_util.py","file_url":"https://github.com/ghliu/mean-field-fcdnn/blob/HEAD/train_acc_util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9f86e91daed794c9"}},{"code_sha256_prefix":"39ce6d260b186572","entry":"d_tanh","repo":"ghliu/mean-field-fcdnn","repo_kind":"official","path":"plot-phase-diagram.py","file_url":"https://github.com/ghliu/mean-field-fcdnn/blob/HEAD/plot-phase-diagram.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"39ce6d260b186572"}},{"code_sha256_prefix":"5bbdf5fd6d349fd2","entry":"eval","repo":"ghliu/mean-field-fcdnn","repo_kind":"official","path":"train_acc_util.py","file_url":"https://github.com/ghliu/mean-field-fcdnn/blob/HEAD/train_acc_util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5bbdf5fd6d349fd2"}},{"code_sha256_prefix":"3725b2e7853d1741","entry":"gauss_density","repo":"ghliu/mean-field-fcdnn","repo_kind":"official","path":"meanfield.py","file_url":"https://github.com/ghliu/mean-field-fcdnn/blob/HEAD/meanfield.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3725b2e7853d1741"}},{"code_sha256_prefix":"46c274b1c87b0944","entry":"generate_fc_dnn","repo":"ghliu/mean-field-fcdnn","repo_kind":"official","path":"train_acc_util.py","file_url":"https://github.com/ghliu/mean-field-fcdnn/blob/HEAD/train_acc_util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"46c274b1c87b0944"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}