{"url":"/method/adaptive-dropout","slug":"adaptive-dropout","name":"Adaptive Dropout","full_name":"Adaptive Dropout","full_name_withheld":false,"description_markdown":"**Adaptive Dropout** is a regularization technique that extends dropout by allowing the dropout probability to be different for different units. The intuition is that there may be hidden units that can individually make confident predictions for the presence or absence of an important feature or combination of features. [Dropout](https://paperswithcode.com/method/dropout) will ignore this confidence and drop the unit out 50% of the time. \r\n\r\nDenote the activity of unit $j$ in a deep neural network by $a\\_{j}$ and assume that its inputs are {$a\\_{i}: i < j$}. In dropout, $a\\_{j}$ is randomly set to zero with probability 0.5. Let $m\\_{j}$ be a binary variable that is used to mask, the activity $a\\_{j}$, so that its value is:\r\n\r\n$$ a\\_{j} = m\\_{j}g \\left( \\sum\\_{i: i