{"url":"/method/shake-shake-regularization","slug":"shake-shake-regularization","name":"Shake-Shake Regularization","full_name":"Shake-Shake Regularization","full_name_withheld":false,"description_markdown":"**Shake-Shake Regularization**  aims to improve the generalization ability of multi-branch networks by replacing the standard summation of parallel branches with a stochastic affine combination. A typical pre-activation [ResNet](https://paperswithcode.com/method/resnet) with 2 residual branches would follow this equation:\r\n\r\n$$x\\_{i+1} = x\\_{i} + \\mathcal{F}\\left(x\\_{i}, \\mathcal{W}\\_{i}^{\\left(1\\right)}\\right) + \\mathcal{F}\\left(x\\_{i}, \\mathcal{W}\\_{i}^{\\left(2\\right)}\\right) $$\r\n\r\nShake-shake regularization introduces a random variable $\\alpha\\_{i}$  following a uniform distribution between 0 and 1 during training:\r\n\r\n$$x\\_{i+1} = x\\_{i} + \\alpha\\mathcal{F}\\left(x\\_{i}, \\mathcal{W}\\_{i}^{\\left(1\\right)}\\right) + \\left(1-\\alpha\\right)\\mathcal{F}\\left(x\\_{i}, \\mathcal{W}\\_{i}^{\\left(2\\right)}\\right) $$\r\n\r\nFollowing the same logic as for [dropout](https://paperswithcode.com/method/dropout), all $\\alpha\\_{i}$ are set to the expected value of $0.5$ at test time.","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":"http://arxiv.org/abs/1705.07485v2","title":"Shake-Shake regularization","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/osmr/imgclsmob/blob/68335927ba27f2356093b985bada0bc3989836b1/pytorch/pytorchcv/models/shakeshakeresnet_cifar.py#L18","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Regularization","url":"/methods/category/regularization","pwc_aliases":[]}],"n_papers_tagged":5,"archive_num_papers":null,"papers_newest_first":[{"paper":"/paper/soft-conditional-computation","title":"CondConv: Conditionally Parameterized Convolutions for Efficient Inference","date":"2019-04-10","arxiv_id":"1904.04971","n_code_links":9,"syntology":null},{"paper":"/paper/data-augmentation-using-random-image-cropping","title":"Data Augmentation using Random Image Cropping and Patching for Deep CNNs","date":"2018-11-22","arxiv_id":"1811.09030","n_code_links":1,"syntology":null},{"paper":"/paper/averaging-weights-leads-to-wider-optima-and","title":"Averaging Weights Leads to Wider Optima and Better Generalization","date":"2018-03-14","arxiv_id":"1803.05407","n_code_links":17,"syntology":{"ran":5,"of":9,"unverified":4,"pointer_only":0}},{"paper":null,"title":"ShakeDrop regularization","date":"2018-01-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/shake-shake-regularization","title":"Shake-Shake regularization","date":"2017-05-21","arxiv_id":"1705.07485","n_code_links":13,"syntology":null}],"papers_shown":5,"tasks":[{"task":"/task/image-classification","name":"Image Classification","papers":2},{"task":"/task/data-augmentation","name":"Data Augmentation","papers":1},{"task":"/task/classification","name":"General Classification","papers":1},{"task":"/task/image-augmentation","name":"Image Augmentation","papers":1},{"task":"/task/image-cropping","name":"Image Cropping","papers":1},{"task":"/task/object-detection","name":"Object Detection","papers":1},{"task":"/task/retrieval","name":"Retrieval","papers":1},{"task":"/task/stochastic-optimization","name":"Stochastic Optimization","papers":1}],"tasks_shown":8,"n_tasks":8,"usage_by_year":[{"year":"2017","papers":1},{"year":"2018","papers":3},{"year":"2019","papers":1}],"row_source":"embedded","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/shake-shake-regularization"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}