{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/adversarial-training-versus-weight-decay","title":"Adversarial Training Versus Weight Decay","arxiv_id":"1804.03308","date":"2018-04-10","proceeding":null,"authors":["Angus Galloway","Thomas Tanay","Graham W. Taylor"],"abstract":"Performance-critical machine learning models should be robust to input\nperturbations not seen during training. Adversarial training is a method for\nimproving a model's robustness to some perturbations by including them in the\ntraining process, but this tends to exacerbate other vulnerabilities of the\nmodel. The adversarial training framework has the effect of translating the\ndata with respect to the cost function, while weight decay has a scaling\neffect. Although weight decay could be considered a crude regularization\ntechnique, it appears superior to adversarial training as it remains stable\nover a broader range of regimes and reduces all generalization errors. Equipped\nwith these abstractions, we provide key baseline results and methodology for\ncharacterizing robustness. The two approaches can be combined to yield one\nsmall model that demonstrates good robustness to several white-box attacks\nassociated with different metrics.","url_abs":"http://arxiv.org/abs/1804.03308v3","url_pdf":"http://arxiv.org/pdf/1804.03308v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"adversarial-training-versus-weight-decay","repo_url":"https://github.com/AngusG/tflite-android-black-box-attacks","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"adversarial-training-versus-weight-decay","repo_url":"https://github.com/uoguelph-mlrg/adversarial_training_vs_weight_decay","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"unanswered"}}],"tasks":[],"methods":[{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1804.03308","atlas_url":"https://app.syntology.ai/?focus=1804.03308","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}