{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/backdrop-stochastic-backpropagation","title":"Backdrop: Stochastic Backpropagation","arxiv_id":"1806.01337","date":"2018-06-04","proceeding":"ICLR 2019 5","authors":["Siavash Golkar","Kyle Cranmer"],"abstract":"We introduce backdrop, a flexible and simple-to-implement method, intuitively\ndescribed as dropout acting only along the backpropagation pipeline. Backdrop\nis implemented via one or more masking layers which are inserted at specific\npoints along the network. Each backdrop masking layer acts as the identity in\nthe forward pass, but randomly masks parts of the backward gradient\npropagation. Intuitively, inserting a backdrop layer after any convolutional\nlayer leads to stochastic gradients corresponding to features of that scale.\nTherefore, backdrop is well suited for problems in which the data have a\nmulti-scale, hierarchical structure. Backdrop can also be applied to problems\nwith non-decomposable loss functions where standard SGD methods are not well\nsuited. We perform a number of experiments and demonstrate that backdrop leads\nto significant improvements in generalization.","url_abs":"http://arxiv.org/abs/1806.01337v1","url_pdf":"http://arxiv.org/pdf/1806.01337v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"backdrop-stochastic-backpropagation","repo_url":"https://github.com/dexgen/backdrop","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[],"methods":[{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"sgd","method_name":"SGD"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}