{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/invariant-backpropagation-how-to-train-a","title":"Invariant backpropagation: how to train a transformation-invariant neural network","arxiv_id":"1502.04434","date":"2015-02-16","proceeding":null,"authors":["Sergey Demyanov","James Bailey","Ramamohanarao Kotagiri","Christopher Leckie"],"abstract":"In many classification problems a classifier should be robust to small\nvariations in the input vector. This is a desired property not only for\nparticular transformations, such as translation and rotation in image\nclassification problems, but also for all others for which the change is small\nenough to retain the object perceptually indistinguishable. We propose two\nextensions of the backpropagation algorithm that train a neural network to be\nrobust to variations in the feature vector. While the first of them enforces\nrobustness of the loss function to all variations, the second method trains the\npredictions to be robust to a particular variation which changes the loss\nfunction the most. The second methods demonstrates better results, but is\nslightly slower. We analytically compare the proposed algorithm with two the\nmost similar approaches (Tangent BP and Adversarial Training), and propose\ntheir fast versions. In the experimental part we perform comparison of all\nalgorithms in terms of classification accuracy and robustness to noise on MNIST\nand CIFAR-10 datasets. Additionally we analyze how the performance of the\nproposed algorithm depends on the dataset size and data augmentation.","url_abs":"http://arxiv.org/abs/1502.04434v3","url_pdf":"http://arxiv.org/pdf/1502.04434v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"invariant-backpropagation-how-to-train-a","repo_url":"https://github.com/sdemyanov/ConvNet","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}}],"tasks":[{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"data-augmentation","task_name":"Data Augmentation"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"translation","task_name":"Translation"},{"task_slug":"image-classification","task_name":"image-classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}