Methods › General › Regularization › DropConnect
DropConnect
Introduced by Li Wan et al. in Regularization of Neural Networks using DropConnect
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
DropConnect generalizes Dropout by randomly dropping the weights rather than the activations with probability 1-p. DropConnect is similar to Dropout as it introduces dynamic sparsity within the model, but differs in that the sparsity is on the weights W, rather than the output vectors of a layer. In other words, the fully connected layer with DropConnect becomes a sparsely connected layer in which the connections are chosen at random during the training stage. Note that this is not equivalent to setting W to be a fixed sparse matrix during training.
For a DropConnect layer, the output is given as:
r = a ((M * W)v)
Here r is the output of a layer, v is the input to a layer, W are weight parameters, and M is a binary matrix encoding the connection information where Mᵢⱼ ∼Bernoulli(p). Each element of the mask M is drawn independently for each example during training, essentially instantiating a different connectivity for each example seen. Additionally, the biases are also masked out during training.
Source in the archive: Regularization of Neural Networks using DropConnect, a link on cds.nyu.edu (archive link, not checked and not linked: not a paper host this site links to).
Papers archive 2025-07-28
30 shown of 84, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
Advanced Deep Learning Techniques for Analyzing Earnings Call Transcripts: Methodologies and Applications 27 Feb 2025 · 0 repositories · arXiv:2503.01886
-
Dynamic DropConnect: Enhancing Neural Network Robustness through Adaptive Edge Dropping Strategies 27 Feb 2025 · 1 repository · arXiv:2502.19948
-
No Argument Left Behind: Overlapping Chunks for Faster Processing of Arbitrarily Long Legal Texts 24 Oct 2024 · 0 repositories · arXiv:2410.19184
-
Terrain Classification Enhanced with Uncertainty for Space Exploration Robots from Proprioceptive Data 3 Jul 2024 · 0 repositories · arXiv:2407.03241Syntology ran 6 of 7 samples · 1 unverified
-
RICo: Reddit ideological communities 5 Jun 2024 · 1 repository
-
Information-Theoretic Generalization Bounds for Deep Neural Networks 4 Apr 2024 · 0 repositories · arXiv:2404.03176
-
Exploring Multi-Level Threats in Telegram Data with AI-Human Annotation: A Preliminary Study 15 Dec 2023 · 0 repositories
-
Illicit Darkweb Classification via Natural-language Processing: Classifying Illicit Content of Webpages based on Textual Information 8 Dec 2023 · 0 repositories · arXiv:2312.04944
-
Bayesian posterior approximation with stochastic ensembles 15 Dec 2022 · 1 repository · arXiv:2212.08123Syntology ran 1 of 4 samples · 3 unverified
-
Disentangled Uncertainty and Out of Distribution Detection in Medical Generative Models 11 Nov 2022 · 0 repositories · arXiv:2211.06250
-
Explainable and High-Performance Hate and Offensive Speech Detection 26 Jun 2022 · 0 repositories · arXiv:2206.12983
-
SoftDropConnect (SDC) -- Effective and Efficient Quantification of the Network Uncertainty in Deep MR Image Analysis 20 Jan 2022 · 1 repository · arXiv:2201.08418
-
IIITT@Dravidian-CodeMix-FIRE2021: Transliterate or translate? Sentiment analysis of code-mixed text in Dravidian languages 15 Nov 2021 · 1 repository · arXiv:2111.07906
-
Entropy optimized semi-supervised decomposed vector-quantized variational autoencoder model based on transfer learning for multiclass text classification and generation 10 Nov 2021 · 0 repositories · arXiv:2111.08453
-
Offensive Language Identification in Low-resourced Code-mixed Dravidian languages using Pseudo-labeling 27 Aug 2021 · 1 repository · arXiv:2108.12177
-
Towards Offensive Language Identification for Tamil Code-Mixed YouTube Comments and Posts 24 Aug 2021 · 1 repository · arXiv:2108.10939
-
Learning ULMFiT and Self-Distillation with Calibration for Medical Dialogue System 20 Jul 2021 · 0 repositories · arXiv:2107.09625
-
Structured DropConnect for Uncertainty Inference in Image Classification 16 Jun 2021 · 1 repository · arXiv:2106.08624
-
The Regularizing Effect of Different Output Layer Designs in Deep Neural Networks 21 May 2021 · 0 repositories
-
WHOSe Heritage: Classification of UNESCO World Heritage "Outstanding Universal Value" Documents with Soft Labels 12 Apr 2021 · 1 repository · arXiv:2104.05547
-
L3CubeMahaSent: A Marathi Tweet-based Sentiment Analysis Dataset 21 Mar 2021 · 1 repository · arXiv:2103.11408
-
indicnlp@kgp at DravidianLangTech-EACL2021: Offensive Language Identification in Dravidian Languages 14 Feb 2021 · 1 repository · arXiv:2102.07150
-
indicnlp@ kgp at DravidianLangTech-EACL2021: Offensive Language Identification in Dravidian Languages 14 Feb 2021 · 1 repository
-
Train your classifier first: Cascade Neural Networks Training from upper layers to lower layers 9 Feb 2021 · 0 repositories · arXiv:2102.04697
-
Experimental Evaluation of Deep Learning models for Marathi Text Classification 13 Jan 2021 · 0 repositories · arXiv:2101.04899
-
LaDiff ULMFiT: A Layer Differentiated training approach for ULMFiT 13 Jan 2021 · 1 repository · arXiv:2101.04965
-
Post-Training Weighted Quantization of Neural Networks for Language Models 1 Jan 2021 · 0 repositories
-
HinglishNLP at SemEval-2020 Task 9: Fine-tuned Language Models for Hinglish Sentiment Detection 1 Dec 2020 · 2 repositories
-
Smash at SemEval-2020 Task 7: Optimizing the Hyperparameters of ERNIE 2.0 for Humor Ranking and Rating 1 Dec 2020 · 0 repositories
-
Palomino-Ochoa at SemEval-2020 Task 9: Robust System based on Transformer for Code-Mixed Sentiment Classification 18 Nov 2020 · 0 repositories · arXiv:2011.09448
Tasks archive 2025-07-28
20 shown of 93 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections