{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/monte-carlo-gradient-estimation-in-machine","title":"Monte Carlo Gradient Estimation in Machine Learning","arxiv_id":"1906.10652","date":"2019-06-25","proceeding":null,"authors":["Shakir Mohamed","Mihaela Rosca","Michael Figurnov","andriy mnih"],"abstract":"This paper is a broad and accessible survey of the methods we have at our disposal for Monte Carlo gradient estimation in machine learning and across the statistical sciences: the problem of computing the gradient of an expectation of a function with respect to parameters defining the distribution that is integrated; the problem of sensitivity analysis. In machine learning research, this gradient problem lies at the core of many learning problems, in supervised, unsupervised and reinforcement learning. We will generally seek to rewrite such gradients in a form that allows for Monte Carlo estimation, allowing them to be easily and efficiently used and analysed. We explore three strategies--the pathwise, score function, and measure-valued gradient estimators--exploring their historical development, derivation, and underlying assumptions. We describe their use in other fields, show how they are related and can be combined, and expand on their possible generalisations. Wherever Monte Carlo gradient estimators have been derived and deployed in the past, important advances have followed. A deeper and more widely-held understanding of this problem will lead to further advances, and it is these advances that we wish to support.","url_abs":"https://arxiv.org/abs/1906.10652v2","url_pdf":"https://arxiv.org/pdf/1906.10652v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"monte-carlo-gradient-estimation-in-machine","repo_url":"https://github.com/deepmind/mc_gradients","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"monte-carlo-gradient-estimation-in-machine","repo_url":"https://github.com/sbodenstein/Tutorials","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"machine-learning","task_name":"BIG-bench Machine Learning"},{"task_slug":"reinforcement-learning","task_name":"Reinforcement Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1906.10652","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1906.10652"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/deepmind/mc_gradients","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sbodenstein/Tutorials","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":1,"unverified":7},"by_repo_kind":{"official":{"samples":8,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5a657fd9d0832b35","entry":"get_shape_list","repo":"deepmind/mc_gradients","repo_kind":"official","path":"monte_carlo_gradients/utils.py","file_url":"https://github.com/deepmind/mc_gradients/blob/HEAD/monte_carlo_gradients/utils.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5a657fd9d0832b35"}},{"code_sha256_prefix":"572943904592255d","entry":"get_sklearn_data","repo":"deepmind/mc_gradients","repo_kind":"official","path":"monte_carlo_gradients/data_utils.py","file_url":"https://github.com/deepmind/mc_gradients/blob/HEAD/monte_carlo_gradients/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"572943904592255d"}},{"code_sha256_prefix":"3bc272af73832393","entry":"get_sklearn_data_as_tensors","repo":"deepmind/mc_gradients","repo_kind":"official","path":"monte_carlo_gradients/data_utils.py","file_url":"https://github.com/deepmind/mc_gradients/blob/HEAD/monte_carlo_gradients/data_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3bc272af73832393"}},{"code_sha256_prefix":"62947de8cdee14bb","entry":"jacobians","repo":"deepmind/mc_gradients","repo_kind":"official","path":"monte_carlo_gradients/utils.py","file_url":"https://github.com/deepmind/mc_gradients/blob/HEAD/monte_carlo_gradients/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"62947de8cdee14bb"}},{"code_sha256_prefix":"6c49272bcffe1015","entry":"linear_model","repo":"deepmind/mc_gradients","repo_kind":"official","path":"monte_carlo_gradients/bayes_lr.py","file_url":"https://github.com/deepmind/mc_gradients/blob/HEAD/monte_carlo_gradients/bayes_lr.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6c49272bcffe1015"}},{"code_sha256_prefix":"6b1132d50ea05b72","entry":"pathwise_loss","repo":"deepmind/mc_gradients","repo_kind":"official","path":"monte_carlo_gradients/gradient_estimators.py","file_url":"https://github.com/deepmind/mc_gradients/blob/HEAD/monte_carlo_gradients/gradient_estimators.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6b1132d50ea05b72"}},{"code_sha256_prefix":"4cf15008fb71436c","entry":"std_gaussian_from_std_dsmaxwell","repo":"deepmind/mc_gradients","repo_kind":"official","path":"monte_carlo_gradients/dist_utils.py","file_url":"https://github.com/deepmind/mc_gradients/blob/HEAD/monte_carlo_gradients/dist_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4cf15008fb71436c"}},{"code_sha256_prefix":"8691f61b76604cbb","entry":"tile_second_to_last_dim","repo":"deepmind/mc_gradients","repo_kind":"official","path":"monte_carlo_gradients/utils.py","file_url":"https://github.com/deepmind/mc_gradients/blob/HEAD/monte_carlo_gradients/utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8691f61b76604cbb"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}