{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/efficient-learning-of-generative-models-via","title":"Efficient Learning of Generative Models via Finite-Difference Score Matching","arxiv_id":"2007.03317","date":"2020-07-07","proceeding":"NeurIPS 2020 12","authors":["Tianyu Pang","Kun Xu","Chongxuan Li","Yang song","Stefano Ermon","Jun Zhu"],"abstract":"Several machine learning applications involve the optimization of higher-order derivatives (e.g., gradients of gradients) during training, which can be expensive in respect to memory and computation even with automatic differentiation. As a typical example in generative modeling, score matching (SM) involves the optimization of the trace of a Hessian. To improve computing efficiency, we rewrite the SM objective and its variants in terms of directional derivatives, and present a generic strategy to efficiently approximate any-order directional derivative with finite difference (FD). Our approximation only involves function evaluations, which can be executed in parallel, and no gradient computations. Thus, it reduces the total computational cost while also improving numerical stability. We provide two instantiations by reformulating variants of SM objectives into the FD forms. Empirically, we demonstrate that our methods produce results comparable to the gradient-based counterparts while being much more computationally efficient.","url_abs":"https://arxiv.org/abs/2007.03317v2","url_pdf":"https://arxiv.org/pdf/2007.03317v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"efficient-learning-of-generative-models-via","repo_url":"https://github.com/taufikxu/FD-ScoreMatching","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2007.03317","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2007.03317"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/taufikxu/FD-ScoreMatching","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"unverified":13},"by_repo_kind":{"official":{"samples":14,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1492fccd0aa41223","entry":"dsm","repo":"taufikxu/FD-ScoreMatching","repo_kind":"official","path":"SSM/losses/dsm.py","file_url":"https://github.com/taufikxu/FD-ScoreMatching/blob/HEAD/SSM/losses/dsm.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1492fccd0aa41223"}},{"code_sha256_prefix":"595dc4b579f696d3","entry":"dsm_fd","repo":"taufikxu/FD-ScoreMatching","repo_kind":"official","path":"SSM/losses/efficient_sm.py","file_url":"https://github.com/taufikxu/FD-ScoreMatching/blob/HEAD/SSM/losses/efficient_sm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"595dc4b579f696d3"}},{"code_sha256_prefix":"874e2e5f1e75a832","entry":"dsm_multi_scale","repo":"taufikxu/FD-ScoreMatching","repo_kind":"official","path":"DeepEBM/ESM/loss_demo.py","file_url":"https://github.com/taufikxu/FD-ScoreMatching/blob/HEAD/DeepEBM/ESM/loss_demo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"874e2e5f1e75a832"}},{"code_sha256_prefix":"bd6e572302d366ca","entry":"dsm_tracetrick","repo":"taufikxu/FD-ScoreMatching","repo_kind":"official","path":"SSM/losses/dsm.py","file_url":"https://github.com/taufikxu/FD-ScoreMatching/blob/HEAD/SSM/losses/dsm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bd6e572302d366ca"}},{"code_sha256_prefix":"1bde925eee0e6851","entry":"efficient_score_matching_conjugate","repo":"taufikxu/FD-ScoreMatching","repo_kind":"official","path":"SSM/losses/efficient_sm.py","file_url":"https://github.com/taufikxu/FD-ScoreMatching/blob/HEAD/SSM/losses/efficient_sm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1bde925eee0e6851"}},{"code_sha256_prefix":"a63496399c14609a","entry":"exact_score_matching","repo":"taufikxu/FD-ScoreMatching","repo_kind":"official","path":"DeepEBM/ESM/loss_train.py","file_url":"https://github.com/taufikxu/FD-ScoreMatching/blob/HEAD/DeepEBM/ESM/loss_train.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a63496399c14609a"}},{"code_sha256_prefix":"2fdedceeec3a9bd6","entry":"exact_score_matching_old","repo":"taufikxu/FD-ScoreMatching","repo_kind":"official","path":"SSM/losses/score_matching.py","file_url":"https://github.com/taufikxu/FD-ScoreMatching/blob/HEAD/SSM/losses/score_matching.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2fdedceeec3a9bd6"}},{"code_sha256_prefix":"6b5f231263f6eb40","entry":"reshape_factor","repo":"taufikxu/FD-ScoreMatching","repo_kind":"official","path":"DeepEBM/ESM/model_condition.py","file_url":"https://github.com/taufikxu/FD-ScoreMatching/blob/HEAD/DeepEBM/ESM/model_condition.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6b5f231263f6eb40"}},{"code_sha256_prefix":"7e568b8e4b4cc379","entry":"score_matching","repo":"taufikxu/FD-ScoreMatching","repo_kind":"official","path":"DeepEBM/ESM/loss_demo.py","file_url":"https://github.com/taufikxu/FD-ScoreMatching/blob/HEAD/DeepEBM/ESM/loss_demo.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7e568b8e4b4cc379"}},{"code_sha256_prefix":"6a824135346a2418","entry":"score_matching","repo":"taufikxu/FD-ScoreMatching","repo_kind":"official","path":"SSM/losses/score_matching.py","file_url":"https://github.com/taufikxu/FD-ScoreMatching/blob/HEAD/SSM/losses/score_matching.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6a824135346a2418"}},{"code_sha256_prefix":"0f4e80bfcddbb1dc","entry":"single_efficient_score_matching","repo":"taufikxu/FD-ScoreMatching","repo_kind":"official","path":"SSM/losses/efficient_sm.py","file_url":"https://github.com/taufikxu/FD-ScoreMatching/blob/HEAD/SSM/losses/efficient_sm.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0f4e80bfcddbb1dc"}},{"code_sha256_prefix":"3b5b96bba781a82e","entry":"soft_binary_cross_entropy","repo":"taufikxu/FD-ScoreMatching","repo_kind":"official","path":"DeepEBM/Torture/loss_function.py","file_url":"https://github.com/taufikxu/FD-ScoreMatching/blob/HEAD/DeepEBM/Torture/loss_function.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3b5b96bba781a82e"}},{"code_sha256_prefix":"fbd5704657ae5a16","entry":"soft_cross_entropy","repo":"taufikxu/FD-ScoreMatching","repo_kind":"official","path":"DeepEBM/Torture/loss_function.py","file_url":"https://github.com/taufikxu/FD-ScoreMatching/blob/HEAD/DeepEBM/Torture/loss_function.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fbd5704657ae5a16"}},{"code_sha256_prefix":"a6715e5746275de0","entry":"soft_cross_entropy_sparse","repo":"taufikxu/FD-ScoreMatching","repo_kind":"official","path":"DeepEBM/Torture/loss_function.py","file_url":"https://github.com/taufikxu/FD-ScoreMatching/blob/HEAD/DeepEBM/Torture/loss_function.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a6715e5746275de0"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}