{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/adaptive-neural-connections-for-sparsity","title":"Adaptive Neural Connections for Sparsity Learning","arxiv_id":null,"date":"2020-03-05","proceeding":"The IEEE Winter Conference on Applications of Computer Vision (WACV), 2020 2020 3","authors":["Prakhar Kaushik","Alex Gain","Hava Siegelmann"],"abstract":"Sparsity learning aims to decrease the computational and memory costs of large deep neural networks (DNNs) via pruning neural connections while simultaneously retaining high accuracy. A large body of work has developed sparsity learning approaches, with recent large-scale experiments showing that two main methods, magnitude pruning and Variational Dropout (VD), achieve similar state-of-the-art results for classification tasks. We propose Adaptive Neural Connections (ANC), a method for explicitly parameterizing fine-grained neuron-to-neuron connections via adjacency matrices at each layer that are learned through backpropagation. Explicitly parameterizing neuron-to-neuron connections confers two primary advantages: 1. Sparsity can be explicitly optimized for via norm-based regularization on the adjacency matrices; and 2. When combined with VD (which we term, ANC-VD), the adjacencies can be interpreted as learned weight importance parameters, which we hypothesize leads to improved convergence for VD. Experiments with ResNet18 show that architectures augmented with ANC outperform their vanilla counterparts.","url_abs":"http://openaccess.thecvf.com/content_WACV_2020/html/Gain_Adaptive_Neural_Connections_for_Sparsity_Learning_WACV_2020_paper.html","url_pdf":"http://openaccess.thecvf.com/content_WACV_2020/papers/Gain_Adaptive_Neural_Connections_for_Sparsity_Learning_WACV_2020_paper.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"model-compression","task_name":"Model Compression"},{"task_slug":"network-pruning","task_name":"Network Pruning"},{"task_slug":"sparse-learning","task_name":"Sparse Learning"}],"methods":[{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"pruning","method_name":"Pruning"},{"method_slug":"variational-dropout","method_name":"Variational Dropout"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/sparse-learning-on-cinic-10-1","task":"Sparse Learning","dataset":"CINIC-10","model":"Resnet18","rank_in_archive_order":1,"of":1,"metrics":{"Sparsity":"92.43"},"uses_additional_data":false},{"leaderboard":"/sota/sparse-learning-on-imagenet32-1","task":"Sparse Learning","dataset":"ImageNet32","model":"Resnet18","rank_in_archive_order":1,"of":1,"metrics":{"Sparsity":"93.63"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}