{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/polygonal-unadjusted-langevin-algorithms","title":"Polygonal Unadjusted Langevin Algorithms: Creating stable and efficient adaptive algorithms for neural networks","arxiv_id":"2105.13937","date":"2021-05-28","proceeding":null,"authors":["Dong-Young Lim","Sotirios Sabanis"],"abstract":"We present a new class of Langevin based algorithms, which overcomes many of the known shortcomings of popular adaptive optimizers that are currently used for the fine tuning of deep learning models. Its underpinning theory relies on recent advances of Euler's polygonal approximations for stochastic differential equations (SDEs) with monotone coefficients. As a result, it inherits the stability properties of tamed algorithms, while it addresses other known issues, e.g. vanishing gradients in neural networks. In particular, we provide a nonasymptotic analysis and full theoretical guarantees for the convergence properties of an algorithm of this novel class, which we named TH$\\varepsilon$O POULA (or, simply, TheoPouLa). Finally, several experiments are presented with different types of deep learning models, which show the superior performance of TheoPouLa over many popular adaptive optimization algorithms.","url_abs":"https://arxiv.org/abs/2105.13937v3","url_pdf":"https://arxiv.org/pdf/2105.13937v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"polygonal-unadjusted-langevin-algorithms","repo_url":"https://github.com/DongyoungLim/THEO_POULA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"polygonal-unadjusted-langevin-algorithms","repo_url":"https://github.com/fin-young/THEO_POULA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"},{"task_slug":"stochastic-optimization","task_name":"Stochastic Optimization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2105.13937","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2105.13937"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/fin-young/THEO_POULA","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/DongyoungLim/THEO_POULA","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":2,"ran_draft_wrong":2,"unverified":7},"by_repo_kind":{"listed":{"samples":11,"ran":4,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"dfb395bae3d07d76","entry":"DenseNet121","repo":"DongyoungLim/THEO_POULA","repo_kind":"listed","path":"models/densenet.py","file_url":"https://github.com/DongyoungLim/THEO_POULA/blob/HEAD/models/densenet.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dfb395bae3d07d76"}},{"code_sha256_prefix":"1f32d831b2c4c04b","entry":"batchify","repo":"fin-young/THEO_POULA","repo_kind":"listed","path":"train_lstm.py","file_url":"https://github.com/fin-young/THEO_POULA/blob/HEAD/train_lstm.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1f32d831b2c4c04b"}},{"code_sha256_prefix":"ac62432dc5134b0d","entry":"make_layers","repo":"DongyoungLim/THEO_POULA","repo_kind":"listed","path":"models/vgg.py","file_url":"https://github.com/DongyoungLim/THEO_POULA/blob/HEAD/models/vgg.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ac62432dc5134b0d"}},{"code_sha256_prefix":"d007cafd2da71575","entry":"repackage_hidden","repo":"fin-young/THEO_POULA","repo_kind":"listed","path":"train_lstm.py","file_url":"https://github.com/fin-young/THEO_POULA/blob/HEAD/train_lstm.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d007cafd2da71575"}},{"code_sha256_prefix":"ac3dd06d1a08c549","entry":"ResNet34","repo":"DongyoungLim/THEO_POULA","repo_kind":"listed","path":"models/resnet.py","file_url":"https://github.com/DongyoungLim/THEO_POULA/blob/HEAD/models/resnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ac3dd06d1a08c549"}},{"code_sha256_prefix":"3725e679e2498aa7","entry":"bin_centers_of_mass","repo":"DongyoungLim/THEO_POULA","repo_kind":"listed","path":"metrics_ilb.py","file_url":"https://github.com/DongyoungLim/THEO_POULA/blob/HEAD/metrics_ilb.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3725e679e2498aa7"}},{"code_sha256_prefix":"fea7bb86b06f3fd5","entry":"bin_predictions_and_accuracies","repo":"DongyoungLim/THEO_POULA","repo_kind":"listed","path":"metrics_ilb.py","file_url":"https://github.com/DongyoungLim/THEO_POULA/blob/HEAD/metrics_ilb.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fea7bb86b06f3fd5"}},{"code_sha256_prefix":"9fca78c7f571171f","entry":"expected_calibration_error","repo":"DongyoungLim/THEO_POULA","repo_kind":"listed","path":"metrics_ilb.py","file_url":"https://github.com/DongyoungLim/THEO_POULA/blob/HEAD/metrics_ilb.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9fca78c7f571171f"}},{"code_sha256_prefix":"ecea09f59d8500ab","entry":"get_batch","repo":"fin-young/THEO_POULA","repo_kind":"listed","path":"train_lstm.py","file_url":"https://github.com/fin-young/THEO_POULA/blob/HEAD/train_lstm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"ecea09f59d8500ab"}},{"code_sha256_prefix":"e4d45132558d0903","entry":"vgg11","repo":"DongyoungLim/THEO_POULA","repo_kind":"listed","path":"models/vgg.py","file_url":"https://github.com/DongyoungLim/THEO_POULA/blob/HEAD/models/vgg.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e4d45132558d0903"}},{"code_sha256_prefix":"b98435ae05182b25","entry":"vgg11_bn","repo":"DongyoungLim/THEO_POULA","repo_kind":"listed","path":"models/vgg.py","file_url":"https://github.com/DongyoungLim/THEO_POULA/blob/HEAD/models/vgg.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b98435ae05182b25"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}