{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-expander-networks-efficient-deep","title":"Deep Expander Networks: Efficient Deep Networks from Graph Theory","arxiv_id":"1711.08757","date":"2017-11-23","proceeding":"ECCV 2018 9","authors":["Ameya Prabhu","Girish Varma","Anoop Namboodiri"],"abstract":"Efficient CNN designs like ResNets and DenseNet were proposed to improve\naccuracy vs efficiency trade-offs. They essentially increased the connectivity,\nallowing efficient information flow across layers. Inspired by these\ntechniques, we propose to model connections between filters of a CNN using\ngraphs which are simultaneously sparse and well connected. Sparsity results in\nefficiency while well connectedness can preserve the expressive power of the\nCNNs. We use a well-studied class of graphs from theoretical computer science\nthat satisfies these properties known as Expander graphs. Expander graphs are\nused to model connections between filters in CNNs to design networks called\nX-Nets. We present two guarantees on the connectivity of X-Nets: Each node\ninfluences every node in a layer in logarithmic steps, and the number of paths\nbetween two sets of nodes is proportional to the product of their sizes. We\nalso propose efficient training and inference algorithms, making it possible to\ntrain deeper and wider X-Nets effectively.\n  Expander based models give a 4% improvement in accuracy on MobileNet over\ngrouped convolutions, a popular technique, which has the same sparsity but\nworse connectivity. X-Nets give better performance trade-offs than the original\nResNet and DenseNet-BC architectures. We achieve model sizes comparable to\nstate-of-the-art pruning techniques using our simple architecture design,\nwithout any pruning. We hope that this work motivates other approaches to\nutilize results from graph theory to develop efficient network architectures.","url_abs":"http://arxiv.org/abs/1711.08757v3","url_pdf":"http://arxiv.org/pdf/1711.08757v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-expander-networks-efficient-deep","repo_url":"https://github.com/DrImpossible/Deep-Expander-Networks","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"deep-expander-networks-efficient-deep","repo_url":"https://github.com/osmr/imgclsmob","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"mxnet","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"concatenated-skip-connection","method_name":"Concatenated Skip Connection"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-block","method_name":"Dense Block"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"pruning","method_name":"Pruning"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1711.08757","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}