{"url":"/method/stochastic-depth","slug":"stochastic-depth","name":"Stochastic Depth","full_name":"Stochastic Depth","full_name_withheld":false,"description_markdown":"**Stochastic Depth** aims to shrink the depth of a network during training, while\r\nkeeping it unchanged during testing. This is achieved by randomly dropping entire [ResBlocks](https://paperswithcode.com/method/residual-block) during training and bypassing their transformations through skip connections. \r\n\r\nLet $b\\_{l} \\in$ {$0, 1$} denote a Bernoulli random variable, which indicates whether the $l$th ResBlock is active ($b\\_{l} = 1$) or inactive ($b\\_{l} = 0$). Further, let us denote the “survival” probability of ResBlock $l$ as $p\\_{l} = \\text{Pr}\\left(b\\_{l} = 1\\right)$. With this definition we can bypass the $l$th ResBlock by multiplying its function $f\\_{l}$ with $b\\_{l}$ and we extend the update rule to:\r\n\r\n$$ H\\_{l} = \\text{ReLU}\\left(b\\_{l}f\\_{l}\\left(H\\_{l-1}\\right) + \\text{id}\\left(H\\_{l-1}\\right)\\right) $$\r\n\r\nIf $b\\_{l} = 1$, this reduces to the original [ResNet](https://paperswithcode.com/method/resnet) update and this ResBlock remains unchanged. If $b\\_{l} = 0$, the ResBlock reduces to the identity function, $H\\_{l} = \\text{id}\\left((H\\_{l}−1\\right)$.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Deep Networks with Stochastic Depth","paper":"/paper/deep-networks-with-stochastic-depth","first_author":"Gao Huang","n_authors":5,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/deep-networks-with-stochastic-depth"},"source":{"url":"http://arxiv.org/abs/1603.09382v3","title":"Deep Networks with Stochastic Depth","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/osmr/imgclsmob/blob/69d63f99a4929d61a98e0975a4ab45b554835b9e/gluon/gluoncv2/models/resdropresnet_cifar.py#L17","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Regularization","url":"/methods/category/regularization","pwc_aliases":[]}],"n_papers_tagged":463,"archive_num_papers":463,"papers_newest_first":[{"paper":"/paper/2506-08357","title":"MD-ViSCo: A Unified Model for Multi-Directional Vital Sign Waveform Conversion","date":"2025-06-10","arxiv_id":"2506.08357","n_code_links":1,"syntology":null},{"paper":null,"title":"MedMoE: Modality-Specialized Mixture of Experts for Medical Vision-Language Understanding","date":"2025-06-10","arxiv_id":"2506.08356","n_code_links":0,"syntology":null},{"paper":null,"title":"Multi-modal brain MRI synthesis based on SwinUNETR","date":"2025-06-03","arxiv_id":"2506.02467","n_code_links":0,"syntology":null},{"paper":"/paper/sst-self-training-with-self-adaptive-1","title":"SST: Self-training with Self-adaptive Thresholding for Semi-supervised Learning","date":"2025-05-31","arxiv_id":"2506.00467","n_code_links":0,"syntology":null},{"paper":"/paper/deep-modeling-and-optimization-of-medical","title":"Deep Modeling and Optimization of Medical Image Classification","date":"2025-05-29","arxiv_id":"2505.23040","n_code_links":1,"syntology":null},{"paper":"/paper/zipa-a-family-of-efficient-models-for","title":"ZIPA: A family of efficient models for multilingual phone recognition","date":"2025-05-29","arxiv_id":"2505.23170","n_code_links":1,"syntology":{"ran":0,"of":1,"unverified":1,"pointer_only":0}},{"paper":"/paper/agrifm-a-multi-source-temporal-remote-sensing","title":"AgriFM: A Multi-source Temporal Remote Sensing Foundation Model for Crop Mapping","date":"2025-05-27","arxiv_id":"2505.21357","n_code_links":1,"syntology":null},{"paper":null,"title":"Structured Initialization for Vision Transformers","date":"2025-05-26","arxiv_id":"2505.19985","n_code_links":0,"syntology":null},{"paper":null,"title":"Explainable Anatomy-Guided AI for Prostate MRI: Foundation Models and In Silico Clinical Trials for Virtual Biopsy-based Risk Assessment","date":"2025-05-23","arxiv_id":"2505.17971","n_code_links":0,"syntology":null},{"paper":null,"title":"Leveraging Stochastic Depth Training for Adaptive Inference","date":"2025-05-23","arxiv_id":"2505.17626","n_code_links":0,"syntology":null},{"paper":null,"title":"Fusion of Foundation and Vision Transformer Model Features for Dermatoscopic Image Classification","date":"2025-05-22","arxiv_id":"2505.16338","n_code_links":0,"syntology":null},{"paper":null,"title":"Swin Transformer for Robust CGI Images Detection: Intra- and Inter-Dataset Analysis across Multiple Color Spaces","date":"2025-05-22","arxiv_id":"2505.16253","n_code_links":0,"syntology":null},{"paper":null,"title":"Multi-Channel Swin Transformer Framework for Bearing Remaining Useful Life Prediction","date":"2025-05-20","arxiv_id":"2505.14897","n_code_links":0,"syntology":null},{"paper":null,"title":"CheX-DS: Improving Chest X-ray Image Classification with Ensemble Learning Based on DenseNet and Swin Transformer","date":"2025-05-16","arxiv_id":"2505.11168","n_code_links":0,"syntology":null},{"paper":null,"title":"A Deep Learning-Driven Inhalation Injury Grading Assistant Using Bronchoscopy Images","date":"2025-05-13","arxiv_id":"2505.08517","n_code_links":0,"syntology":null},{"paper":null,"title":"Technical Report for ICRA 2025 GOOSE 2D Semantic Segmentation Challenge: Leveraging Color Shift Correction, RoPE-Swin Backbone, and Quantile-based Label Denoising Strategy for Robust Outdoor Scene Understanding","date":"2025-05-11","arxiv_id":"2505.06991","n_code_links":0,"syntology":null},{"paper":"/paper/dfen-dual-feature-equalization-network-for","title":"DFEN: Dual Feature Equalization Network for Medical Image Segmentation","date":"2025-05-09","arxiv_id":"2505.05913","n_code_links":1,"syntology":null},{"paper":null,"title":"Balancing Accuracy, Calibration, and Efficiency in Active Learning with Vision Transformers Under Label Noise","date":"2025-05-07","arxiv_id":"2505.04375","n_code_links":0,"syntology":null},{"paper":null,"title":"SwinLip: An Efficient Visual Speech Encoder for Lip Reading Using Swin Transformer","date":"2025-05-07","arxiv_id":"2505.04394","n_code_links":0,"syntology":null},{"paper":null,"title":"Enhancing DR Classification with Swin Transformer and Shifted Window Attention","date":"2025-04-20","arxiv_id":"2504.15317","n_code_links":0,"syntology":null},{"paper":null,"title":"SatelliteCalculator: A Multi-Task Vision Foundation Model for Quantitative Remote Sensing Inversion","date":"2025-04-18","arxiv_id":"2504.13442","n_code_links":0,"syntology":null},{"paper":"/paper/towards-accurate-and-interpretable","title":"Towards Accurate and Interpretable Neuroblastoma Diagnosis via Contrastive Multi-scale Pathological Image Analysis","date":"2025-04-18","arxiv_id":"2504.13754","n_code_links":1,"syntology":null},{"paper":"/paper/deep-learning-based-bathymetry-retrieval","title":"Deep Learning-based Bathymetry Retrieval without In-situ Depths using Remote Sensing Imagery and SfM-MVS DSMs with Data Gaps","date":"2025-04-15","arxiv_id":"2504.11416","n_code_links":1,"syntology":null},{"paper":null,"title":"HDC: Hierarchical Distillation for Multi-level Noisy Consistency in Semi-Supervised Fetal Ultrasound Segmentation","date":"2025-04-14","arxiv_id":"2504.09876","n_code_links":0,"syntology":null},{"paper":"/paper/foundation-models-for-seismic-data-processing","title":"Foundation Models For Seismic Data Processing: An Extensive Review","date":"2025-03-31","arxiv_id":"2503.24166","n_code_links":1,"syntology":null},{"paper":null,"title":"Camera Model Identification with SPAIR-Swin and Entropy based Non-Homogeneous Patches","date":"2025-03-28","arxiv_id":"2503.22120","n_code_links":0,"syntology":null},{"paper":null,"title":"Context-Aware Semantic Segmentation: Enhancing Pixel-Level Understanding with Large Language Models for Advanced Vision Applications","date":"2025-03-25","arxiv_id":"2503.19276","n_code_links":0,"syntology":null},{"paper":"/paper/inatag-multi-class-classification-models","title":"iNatAg: Multi-Class Classification Models Enabled by a Large-Scale Benchmark Dataset with 4.7M Images of 2,959 Crop and Weed Species","date":"2025-03-25","arxiv_id":"2503.20068","n_code_links":1,"syntology":null},{"paper":"/paper/binarized-mamba-transformer-for-lightweight","title":"Binarized Mamba-Transformer for Lightweight Quad Bayer HybridEVS Demosaicing","date":"2025-03-20","arxiv_id":"2503.16134","n_code_links":1,"syntology":null},{"paper":null,"title":"Alzheimer's Disease Classification Using Retinal OCT: TransnetOCT and Swin Transformer Models","date":"2025-03-14","arxiv_id":"2503.11511","n_code_links":0,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/semantic-segmentation","name":"Semantic Segmentation","papers":94},{"task":"/task/image-classification","name":"Image Classification","papers":65},{"task":"/task/object-detection","name":"Object Detection","papers":52},{"task":"/task/segmentation","name":"Segmentation","papers":51},{"task":"/task/decoder","name":"Decoder","papers":48},{"task":"/task/image-classification","name":"image-classification","papers":44},{"task":"/task/object-detection-1","name":"object-detection","papers":37},{"task":"/task/image-segmentation","name":"Image Segmentation","papers":30},{"task":"/task/instance-segmentation","name":"Instance Segmentation","papers":26},{"task":"/task/data-augmentation","name":"Data Augmentation","papers":24},{"task":"/task/medical-image-segmentation","name":"Medical Image Segmentation","papers":23},{"task":"/task/super-resolution","name":"Super-Resolution","papers":23},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":23},{"task":"/task/classification-1","name":"Classification","papers":21},{"task":"/task/object","name":"Object","papers":21},{"task":"/task/diagnostic","name":"Diagnostic","papers":15},{"task":"/task/self-supervised-learning","name":"Self-Supervised Learning","papers":14},{"task":null,"name":"GPU","papers":13},{"task":"/task/image-super-resolution","name":"Image Super-Resolution","papers":13},{"task":"/task/denoising","name":"Denoising","papers":12}],"tasks_shown":20,"n_tasks":371,"usage_by_year":[{"year":"2016","papers":3},{"year":"2017","papers":1},{"year":"2019","papers":4},{"year":"2020","papers":12},{"year":"2021","papers":57},{"year":"2022","papers":112},{"year":"2023","papers":108},{"year":"2024","papers":113},{"year":"2025","papers":53}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/stochastic-depth"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}