{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/automatic-differentiation-in-machine-learning","title":"Automatic differentiation in machine learning: a survey","arxiv_id":"1502.05767","date":"2015-02-20","proceeding":null,"authors":["Atilim Gunes Baydin","Barak A. Pearlmutter","Alexey Andreyevich Radul","Jeffrey Mark Siskind"],"abstract":"Derivatives, mostly in the form of gradients and Hessians, are ubiquitous in\nmachine learning. Automatic differentiation (AD), also called algorithmic\ndifferentiation or simply \"autodiff\", is a family of techniques similar to but\nmore general than backpropagation for efficiently and accurately evaluating\nderivatives of numeric functions expressed as computer programs. AD is a small\nbut established field with applications in areas including computational fluid\ndynamics, atmospheric sciences, and engineering design optimization. Until very\nrecently, the fields of machine learning and AD have largely been unaware of\neach other and, in some cases, have independently discovered each other's\nresults. Despite its relevance, general-purpose AD has been missing from the\nmachine learning toolbox, a situation slowly changing with its ongoing adoption\nunder the names \"dynamic computational graphs\" and \"differentiable\nprogramming\". We survey the intersection of AD and machine learning, cover\napplications where AD has direct relevance, and address the main implementation\ntechniques. By precisely defining the main differentiation techniques and their\ninterrelationships, we aim to bring clarity to the usage of the terms\n\"autodiff\", \"automatic differentiation\", and \"symbolic differentiation\" as\nthese are encountered more and more in machine learning settings.","url_abs":"http://arxiv.org/abs/1502.05767v4","url_pdf":"http://arxiv.org/pdf/1502.05767v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"automatic-differentiation-in-machine-learning","repo_url":"https://github.com/Giully314/Endurance","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"automatic-differentiation-in-machine-learning","repo_url":"https://github.com/bigaidream-projects/drmad","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"automatic-differentiation-in-machine-learning","repo_url":"https://github.com/gmodena/scalagrad","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"automatic-differentiation-in-machine-learning","repo_url":"https://github.com/ryanrhymes/owl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"machine-learning","task_name":"BIG-bench Machine Learning"},{"task_slug":"survey","task_name":"Survey"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1502.05767","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1502.05767"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/gmodena/scalagrad","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Giully314/Endurance","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ryanrhymes/owl","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/bigaidream-projects/drmad","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":9},"by_repo_kind":{"listed":{"samples":9,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"9aeb785d120a1afa","entry":"datapath","repo":"bigaidream-projects/drmad","repo_kind":"listed","path":"cpu_ver/hypergrad/cifar10.py","file_url":"https://github.com/bigaidream-projects/drmad/blob/HEAD/cpu_ver/hypergrad/cifar10.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9aeb785d120a1afa"}},{"code_sha256_prefix":"8530312518e21c41","entry":"datapath","repo":"bigaidream-projects/drmad","repo_kind":"listed","path":"cpu_ver/hypergrad/data.py","file_url":"https://github.com/bigaidream-projects/drmad/blob/HEAD/cpu_ver/hypergrad/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8530312518e21c41"}},{"code_sha256_prefix":"c77cfd4cb69810a9","entry":"format_as","repo":"bigaidream-projects/drmad","repo_kind":"listed","path":"gpu_ver/hypergrad.py","file_url":"https://github.com/bigaidream-projects/drmad/blob/HEAD/gpu_ver/hypergrad.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c77cfd4cb69810a9"}},{"code_sha256_prefix":"306d1cf5e4cb6fad","entry":"getval","repo":"bigaidream-projects/drmad","repo_kind":"listed","path":"cpu_ver/funkyyak/scheme/core.py","file_url":"https://github.com/bigaidream-projects/drmad/blob/HEAD/cpu_ver/funkyyak/scheme/core.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"306d1cf5e4cb6fad"}},{"code_sha256_prefix":"96e9899611d6d9f6","entry":"load_data","repo":"bigaidream-projects/drmad","repo_kind":"listed","path":"cpu_ver/hypergrad/cifar10.py","file_url":"https://github.com/bigaidream-projects/drmad/blob/HEAD/cpu_ver/hypergrad/cifar10.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"96e9899611d6d9f6"}},{"code_sha256_prefix":"56dfab3bce86044d","entry":"load_data","repo":"bigaidream-projects/drmad","repo_kind":"listed","path":"cpu_ver/hypergrad/data.py","file_url":"https://github.com/bigaidream-projects/drmad/blob/HEAD/cpu_ver/hypergrad/data.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"56dfab3bce86044d"}},{"code_sha256_prefix":"35a6803eda0b78c1","entry":"make_exp_kernel","repo":"bigaidream-projects/drmad","repo_kind":"listed","path":"cpu_ver/hypergrad/kernel_methods.py","file_url":"https://github.com/bigaidream-projects/drmad/blob/HEAD/cpu_ver/hypergrad/kernel_methods.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"35a6803eda0b78c1"}},{"code_sha256_prefix":"9100a3ac53dc7b36","entry":"make_sq_exp_kernel","repo":"bigaidream-projects/drmad","repo_kind":"listed","path":"cpu_ver/hypergrad/kernel_methods.py","file_url":"https://github.com/bigaidream-projects/drmad/blob/HEAD/cpu_ver/hypergrad/kernel_methods.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9100a3ac53dc7b36"}},{"code_sha256_prefix":"028b580dcd973ebd","entry":"weighted_neighbors_loss","repo":"bigaidream-projects/drmad","repo_kind":"listed","path":"cpu_ver/hypergrad/kernel_methods.py","file_url":"https://github.com/bigaidream-projects/drmad/blob/HEAD/cpu_ver/hypergrad/kernel_methods.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"028b580dcd973ebd"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}