{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/preconditioner-on-matrix-lie-group-for-sgd","title":"Preconditioner on Matrix Lie Group for SGD","arxiv_id":"1809.10232","date":"2018-09-26","proceeding":"ICLR 2019 5","authors":["Xi-Lin Li"],"abstract":"We study two types of preconditioners and preconditioned stochastic gradient\ndescent (SGD) methods in a unified framework. We call the first one the Newton\ntype due to its close relationship to the Newton method, and the second one the\nFisher type as its preconditioner is closely related to the inverse of Fisher\ninformation matrix. Both preconditioners can be derived from one framework, and\nefficiently estimated on any matrix Lie groups designated by the user using\nnatural or relative gradient descent minimizing certain preconditioner\nestimation criteria. Many existing preconditioners and methods, e.g., RMSProp,\nAdam, KFAC, equilibrated SGD, batch normalization, etc., are special cases of\nor closely related to either the Newton type or the Fisher type ones.\nExperimental results on relatively large scale machine learning problems are\nreported for performance study.","url_abs":"http://arxiv.org/abs/1809.10232v2","url_pdf":"http://arxiv.org/pdf/1809.10232v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"preconditioner-on-matrix-lie-group-for-sgd","repo_url":"https://github.com/lixilinx/psgd_torch","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"preconditioner-on-matrix-lie-group-for-sgd","repo_url":"https://github.com/lixilinx/psgd_tf","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}}],"tasks":[{"task_slug":"type","task_name":"Vocal Bursts Type Prediction"}],"methods":[{"method_slug":"rmsprop","method_name":"RMSProp"},{"method_slug":"sgd","method_name":"SGD"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1809.10232","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1809.10232"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lixilinx/psgd_tf","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lixilinx/psgd_torch","reach":null}],"summary":{"ran_fixture":1,"ran_honours":1,"ran_draft_wrong":3,"unverified":1},"by_repo_kind":{},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"8d6d14bd8c4c9bc3","entry":"init_kron","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"8d6d14bd8c4c9bc3"}},{"code_sha256_prefix":"eddcb98704cbe648","entry":"lift2single","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"eddcb98704cbe648"}},{"code_sha256_prefix":"e3c58792f0261fb7","entry":"norm_lower_bound_skh","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"e3c58792f0261fb7"}},{"code_sha256_prefix":"61416702d57451ae","entry":"norm_lower_bound_spd","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"61416702d57451ae"}},{"code_sha256_prefix":"cc005e3400dac8f2","entry":"norm_lower_bound_spd","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"cc005e3400dac8f2"}},{"code_sha256_prefix":"17540df6389a595f","entry":"norm_lower_bound_skh","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"17540df6389a595f"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}