{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-neural-network-approach-for-stochastic","title":"A Neural Network Approach for Stochastic Optimal Control","arxiv_id":"2209.13104","date":"2022-09-27","proceeding":null,"authors":["Xingjian Li","Deepanshu Verma","Lars Ruthotto"],"abstract":"We present a neural network approach for approximating the value function of high-dimensional stochastic control problems. Our training process simultaneously updates our value function estimate and identifies the part of the state space likely to be visited by optimal trajectories. Our approach leverages insights from optimal control theory and the fundamental relation between semi-linear parabolic partial differential equations and forward-backward stochastic differential equations. To focus the sampling on relevant states during neural network training, we use the stochastic Pontryagin maximum principle (PMP) to obtain the optimal controls for the current value function estimate. By design, our approach coincides with the method of characteristics for the non-viscous Hamilton-Jacobi-Bellman equation arising in deterministic control problems. Our training loss consists of a weighted sum of the objective functional of the control problem and penalty terms that enforce the HJB equations along the sampled trajectories. Importantly, training is unsupervised in that it does not require solutions of the control problem. Our numerical experiments highlight our scheme's ability to identify the relevant parts of the state space and produce meaningful value estimates. Using a two-dimensional model problem, we demonstrate the importance of the stochastic PMP to inform the sampling and compare to a finite element approach. With a nonlinear control affine quadcopter example, we illustrate that our approach can handle complicated dynamics. For a 100-dimensional benchmark problem, we demonstrate that our approach improves accuracy and time-to-solution and, via a modification, we show the wider applicability of our scheme.","url_abs":"https://arxiv.org/abs/2209.13104v3","url_pdf":"https://arxiv.org/pdf/2209.13104v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"a-neural-network-approach-for-stochastic","repo_url":"https://github.com/emorymlip/fbsnns","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"a-neural-network-approach-for-stochastic","repo_url":"https://github.com/emorymlip/neuralsoc","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2209.13104","atlas_url":"https://app.syntology.ai/?focus=2209.13104","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2209.13104"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/emorymlip/fbsnns","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/emorymlip/neuralsoc","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_honours":2,"ran_fixture":1,"unverified":5},"by_repo_kind":{"official":{"samples":8,"ran":3,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9145a617a5d6aa3a","entry":"antiderivTanh","repo":"emorymlip/neuralsoc","repo_kind":"official","path":"networks.py","file_url":"https://github.com/emorymlip/neuralsoc/blob/HEAD/networks.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9145a617a5d6aa3a"}},{"code_sha256_prefix":"91ae98c615b90e86","entry":"derivTanh","repo":"emorymlip/neuralsoc","repo_kind":"official","path":"Phi_OTflow.py","file_url":"https://github.com/emorymlip/neuralsoc/blob/HEAD/Phi_OTflow.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"91ae98c615b90e86"}},{"code_sha256_prefix":"d25c6435028dd30c","entry":"g","repo":"emorymlip/fbsnns","repo_kind":"official","path":"HamiltonJacobiBellman100D.py","file_url":"https://github.com/emorymlip/fbsnns/blob/HEAD/HamiltonJacobiBellman100D.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d25c6435028dd30c"}},{"code_sha256_prefix":"0877e4d1c6f97cd1","entry":"control_obj","repo":"emorymlip/neuralsoc","repo_kind":"official","path":"loss.py","file_url":"https://github.com/emorymlip/neuralsoc/blob/HEAD/loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0877e4d1c6f97cd1"}},{"code_sha256_prefix":"8a9dca26f4cef296","entry":"get_Phi","repo":"emorymlip/fbsnns","repo_kind":"official","path":"HamiltonJacobiBellman100D.py","file_url":"https://github.com/emorymlip/fbsnns/blob/HEAD/HamiltonJacobiBellman100D.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8a9dca26f4cef296"}},{"code_sha256_prefix":"db0e52aa51f96bd4","entry":"hjb_penalty","repo":"emorymlip/neuralsoc","repo_kind":"official","path":"loss.py","file_url":"https://github.com/emorymlip/neuralsoc/blob/HEAD/loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"db0e52aa51f96bd4"}},{"code_sha256_prefix":"6e1d3a4c50da9847","entry":"terminal_penalty","repo":"emorymlip/neuralsoc","repo_kind":"official","path":"loss.py","file_url":"https://github.com/emorymlip/neuralsoc/blob/HEAD/loss.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6e1d3a4c50da9847"}},{"code_sha256_prefix":"ce4b7a20ab44ad6c","entry":"u_exact","repo":"emorymlip/fbsnns","repo_kind":"official","path":"HamiltonJacobiBellman100D.py","file_url":"https://github.com/emorymlip/fbsnns/blob/HEAD/HamiltonJacobiBellman100D.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ce4b7a20ab44ad6c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}