{"url":"/method/dropconnect","slug":"dropconnect","name":"DropConnect","full_name":"DropConnect","full_name_withheld":false,"description_markdown":"**DropConnect** generalizes [Dropout](https://paperswithcode.com/method/dropout) by randomly dropping the weights rather than the activations with probability $1-p$. DropConnect is similar to Dropout as it introduces dynamic sparsity within the model, but differs in that the sparsity is on the weights $W$, rather than the output vectors of a layer. In other words, the fully connected layer with DropConnect becomes a sparsely connected layer in which the connections are chosen at random during the training stage. Note that this is not equivalent to setting $W$ to be a fixed sparse matrix during training.\r\n\r\nFor a DropConnect layer, the output is given as:\r\n\r\n$$ r = a \\left(\\left(M * W\\right){v}\\right)$$\r\n\r\nHere $r$ is the output of a layer, $v$ is the input to a layer, $W$ are weight parameters, and $M$ is a binary matrix encoding the connection information where $M\\_{ij} \\sim \\text{Bernoulli}\\left(p\\right)$. Each element of the mask $M$ is drawn independently for each example during training, essentially instantiating a different connectivity for each example seen. Additionally, the biases are also masked out during training.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Regularization of Neural Networks using DropConnect","paper":"/paper/regularization-of-neural-networks-using","first_author":"Li Wan","n_authors":5,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/regularization-of-neural-networks-using"},"source":{"url":"http://cds.nyu.edu/projects/regularization-neural-networks-using-dropconnect/","title":"Regularization of Neural Networks using DropConnect","url_on_a_paper_host":false},"code_snippet_url":"https://github.com/teelinsan/KerasDropconnect/blob/0b203b56b2564686358fe0907b89316e3fce4014/ddrop/layers.py#L24","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Regularization","url":"/methods/category/regularization","pwc_aliases":[]}],"n_papers_tagged":84,"archive_num_papers":84,"papers_newest_first":[{"paper":null,"title":"Advanced Deep Learning Techniques for Analyzing Earnings Call Transcripts: Methodologies and Applications","date":"2025-02-27","arxiv_id":"2503.01886","n_code_links":0,"syntology":null},{"paper":"/paper/dynamic-dropconnect-enhancing-neural-network","title":"Dynamic DropConnect: Enhancing Neural Network Robustness through Adaptive Edge Dropping Strategies","date":"2025-02-27","arxiv_id":"2502.19948","n_code_links":1,"syntology":null},{"paper":null,"title":"No Argument Left Behind: Overlapping Chunks for Faster Processing of Arbitrarily Long Legal Texts","date":"2024-10-24","arxiv_id":"2410.19184","n_code_links":0,"syntology":null},{"paper":"/paper/terrain-classification-enhanced-with","title":"Terrain Classification Enhanced with Uncertainty for Space Exploration Robots from Proprioceptive Data","date":"2024-07-03","arxiv_id":"2407.03241","n_code_links":0,"syntology":{"ran":6,"of":7,"unverified":1,"pointer_only":0}},{"paper":"/paper/rico-reddit-ideological-communities","title":"RICo: Reddit ideological communities","date":"2024-06-05","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"title":"Information-Theoretic Generalization Bounds for Deep Neural Networks","date":"2024-04-04","arxiv_id":"2404.03176","n_code_links":0,"syntology":null},{"paper":"/paper/exploring-multi-level-threats-in-telegram","title":"Exploring Multi-Level Threats in Telegram Data with AI-Human Annotation: A Preliminary Study","date":"2023-12-15","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"title":"Illicit Darkweb Classification via Natural-language Processing: Classifying Illicit Content of Webpages based on Textual Information","date":"2023-12-08","arxiv_id":"2312.04944","n_code_links":0,"syntology":null},{"paper":"/paper/bayesian-posterior-approximation-with","title":"Bayesian posterior approximation with stochastic ensembles","date":"2022-12-15","arxiv_id":"2212.08123","n_code_links":1,"syntology":{"ran":1,"of":4,"unverified":3,"pointer_only":0}},{"paper":null,"title":"Disentangled Uncertainty and Out of Distribution Detection in Medical Generative Models","date":"2022-11-11","arxiv_id":"2211.06250","n_code_links":0,"syntology":null},{"paper":null,"title":"Explainable and High-Performance Hate and Offensive Speech Detection","date":"2022-06-26","arxiv_id":"2206.12983","n_code_links":0,"syntology":null},{"paper":"/paper/softdropconnect-sdc-effective-and-efficient","title":"SoftDropConnect (SDC) -- Effective and Efficient Quantification of the Network Uncertainty in Deep MR Image Analysis","date":"2022-01-20","arxiv_id":"2201.08418","n_code_links":1,"syntology":null},{"paper":"/paper/iiitt-dravidian-codemix-fire2021","title":"IIITT@Dravidian-CodeMix-FIRE2021: Transliterate or translate? Sentiment analysis of code-mixed text in Dravidian languages","date":"2021-11-15","arxiv_id":"2111.07906","n_code_links":1,"syntology":null},{"paper":null,"title":"Entropy optimized semi-supervised decomposed vector-quantized variational autoencoder model based on transfer learning for multiclass text classification and generation","date":"2021-11-10","arxiv_id":"2111.08453","n_code_links":0,"syntology":null},{"paper":"/paper/offensive-language-identification-in-low","title":"Offensive Language Identification in Low-resourced Code-mixed Dravidian languages using Pseudo-labeling","date":"2021-08-27","arxiv_id":"2108.12177","n_code_links":1,"syntology":null},{"paper":"/paper/sn-computer-science-towards-offensive","title":"Towards Offensive Language Identification for Tamil Code-Mixed YouTube Comments and Posts","date":"2021-08-24","arxiv_id":"2108.10939","n_code_links":1,"syntology":null},{"paper":null,"title":"Learning ULMFiT and Self-Distillation with Calibration for Medical Dialogue System","date":"2021-07-20","arxiv_id":"2107.09625","n_code_links":0,"syntology":null},{"paper":"/paper/structured-dropconnect-for-uncertainty","title":"Structured DropConnect for Uncertainty Inference in Image Classification","date":"2021-06-16","arxiv_id":"2106.08624","n_code_links":1,"syntology":null},{"paper":null,"title":"The Regularizing Effect of Different Output Layer Designs in Deep Neural Networks","date":"2021-05-21","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/whose-heritage-classification-of-unesco-world","title":"WHOSe Heritage: Classification of UNESCO World Heritage \"Outstanding Universal Value\" Documents with Soft Labels","date":"2021-04-12","arxiv_id":"2104.05547","n_code_links":1,"syntology":null},{"paper":"/paper/l3cubemahasent-a-marathi-tweet-based","title":"L3CubeMahaSent: A Marathi Tweet-based Sentiment Analysis Dataset","date":"2021-03-21","arxiv_id":"2103.11408","n_code_links":1,"syntology":null},{"paper":"/paper/indicnlp-kgp-at-dravidianlangtech-eacl2021","title":"indicnlp@kgp at DravidianLangTech-EACL2021: Offensive Language Identification in Dravidian Languages","date":"2021-02-14","arxiv_id":"2102.07150","n_code_links":1,"syntology":null},{"paper":"/paper/indicnlp-kgp-at-dravidianlangtech-eacl2021-1","title":"indicnlp@ kgp at DravidianLangTech-EACL2021: Offensive Language Identification in Dravidian Languages","date":"2021-02-14","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"title":"Train your classifier first: Cascade Neural Networks Training from upper layers to lower layers","date":"2021-02-09","arxiv_id":"2102.04697","n_code_links":0,"syntology":null},{"paper":null,"title":"Experimental Evaluation of Deep Learning models for Marathi Text Classification","date":"2021-01-13","arxiv_id":"2101.04899","n_code_links":0,"syntology":null},{"paper":"/paper/ladiff-ulmfit-a-layer-differentiated-training","title":"LaDiff ULMFiT: A Layer Differentiated training approach for ULMFiT","date":"2021-01-13","arxiv_id":"2101.04965","n_code_links":1,"syntology":null},{"paper":null,"title":"Post-Training Weighted Quantization of Neural Networks for Language Models","date":"2021-01-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/hinglishnlp-at-semeval-2020-task-9-fine-tuned","title":"HinglishNLP at SemEval-2020 Task 9: Fine-tuned Language Models for Hinglish Sentiment Detection","date":"2020-12-01","arxiv_id":null,"n_code_links":2,"syntology":null},{"paper":null,"title":"Smash at SemEval-2020 Task 7: Optimizing the Hyperparameters of ERNIE 2.0 for Humor Ranking and Rating","date":"2020-12-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"title":"Palomino-Ochoa at SemEval-2020 Task 9: Robust System based on Transformer for Code-Mixed Sentiment Classification","date":"2020-11-18","arxiv_id":"2011.09448","n_code_links":0,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/language-modelling","name":"Language Modelling","papers":20},{"task":"/task/language-modeling","name":"Language Modeling","papers":18},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":18},{"task":"/task/text-classification","name":"Text Classification","papers":16},{"task":"/task/classification","name":"General Classification","papers":15},{"task":"/task/text-classification-1","name":"text-classification","papers":12},{"task":"/task/classification-1","name":"Classification","papers":11},{"task":"/task/sentiment-analysis","name":"Sentiment Analysis","papers":9},{"task":"/task/image-classification","name":"Image Classification","papers":8},{"task":"/task/translation","name":"Translation","papers":6},{"task":"/task/image-classification","name":"image-classification","papers":5},{"task":"/task/language-identification","name":"Language Identification","papers":4},{"task":"/task/machine-translation","name":"Machine Translation","papers":4},{"task":"/task/word-embeddings","name":"Word Embeddings","papers":4},{"task":"/task/bayesian-inference","name":"Bayesian Inference","papers":3},{"task":"/task/decision-making","name":"Decision Making","papers":3},{"task":"/task/hate-speech-detection","name":"Hate Speech Detection","papers":3},{"task":"/task/sentence","name":"Sentence","papers":3},{"task":"/task/sentiment-classification","name":"Sentiment Classification","papers":3},{"task":"/task/time-series-1","name":"Time Series","papers":3}],"tasks_shown":20,"n_tasks":93,"usage_by_year":[{"year":"2013","papers":2},{"year":"2015","papers":2},{"year":"2016","papers":3},{"year":"2017","papers":3},{"year":"2018","papers":6},{"year":"2019","papers":20},{"year":"2020","papers":21},{"year":"2021","papers":15},{"year":"2022","papers":4},{"year":"2023","papers":2},{"year":"2024","papers":4},{"year":"2025","papers":2}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/dropconnect"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}