{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/spinalnet-deep-neural-network-with-gradual-1","title":"SpinalNet: Deep Neural Network with Gradual Input","arxiv_id":"2007.03347","date":"2020-07-07","proceeding":"arXiv 2020 7","authors":["H M Dipu Kabir","Moloud Abdar","Seyed Mohammad Jafar Jalali","Abbas Khosravi","Amir F. Atiya","Saeid Nahavandi","Dipti Srinivasan"],"abstract":"Deep neural networks (DNNs) have achieved the state of the art performance in numerous fields. However, DNNs need high computation times, and people always expect better performance in a lower computation. Therefore, we study the human somatosensory system and design a neural network (SpinalNet) to achieve higher accuracy with fewer computations. Hidden layers in traditional NNs receive inputs in the previous layer, apply activation function, and then transfer the outcomes to the next layer. In the proposed SpinalNet, each layer is split into three splits: 1) input split, 2) intermediate split, and 3) output split. Input split of each layer receives a part of the inputs. The intermediate split of each layer receives outputs of the intermediate split of the previous layer and outputs of the input split of the current layer. The number of incoming weights becomes significantly lower than traditional DNNs. The SpinalNet can also be used as the fully connected or classification layer of DNN and supports both traditional learning and transfer learning. We observe significant error reductions with lower computational costs in most of the DNNs. Traditional learning on the VGG-5 network with SpinalNet classification layers provided the state-of-the-art (SOTA) performance on QMNIST, Kuzushiji-MNIST, EMNIST (Letters, Digits, and Balanced) datasets. Traditional learning with ImageNet pre-trained initial weights and SpinalNet classification layers provided the SOTA performance on STL-10, Fruits 360, Bird225, and Caltech-101 datasets. The scripts of the proposed SpinalNet are available at the following link: https://github.com/dipuk0506/SpinalNet","url_abs":"https://arxiv.org/abs/2007.03347v3","url_pdf":"https://arxiv.org/pdf/2007.03347v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"spinalnet-deep-neural-network-with-gradual-1","repo_url":"https://github.com/dipuk0506/SpinalNet","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"spinalnet-deep-neural-network-with-gradual-1","repo_url":"https://github.com/Mechachleopteryx/SpinalNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"spinalnet-deep-neural-network-with-gradual-1","repo_url":"https://github.com/dipuk0506/uq","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"fine-grained-image-classification","task_name":"Fine-Grained Image Classification"},{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"satellite-image-classification","task_name":"Satellite Image Classification"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[{"method_slug":"adam","method_name":"Adam"},{"method_slug":"image-scale-augmentation","method_name":"Image Scale Augmentation"},{"method_slug":"randaugment","method_name":"RandAugment"},{"method_slug":"sgd-with-momentum","method_name":"SGD with Momentum"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/fine-grained-image-classification-on-bird-225","task":"Fine-Grained Image Classification","dataset":"Bird-225","model":"VGG-19bn (Spinal FC)","rank_in_archive_order":4,"of":5,"metrics":{"Accuracy":"99.02"},"uses_additional_data":true},{"leaderboard":"/sota/fine-grained-image-classification-on-bird-225","task":"Fine-Grained Image Classification","dataset":"Bird-225","model":"VGG-19bn","rank_in_archive_order":5,"of":5,"metrics":{"Accuracy":"98.67"},"uses_additional_data":true},{"leaderboard":"/sota/fine-grained-image-classification-on-caltech","task":"Fine-Grained Image Classification","dataset":"Caltech-101","model":"Wide-ResNet-101 (Spinal FC)","rank_in_archive_order":2,"of":18,"metrics":{"Accuracy":"97.32","Top-1 Error Rate":"2.68%"},"uses_additional_data":true},{"leaderboard":"/sota/fine-grained-image-classification-on-caltech","task":"Fine-Grained Image Classification","dataset":"Caltech-101","model":"Wide-ResNet-101","rank_in_archive_order":3,"of":18,"metrics":{"Top-1 Error Rate":"2.89%"},"uses_additional_data":true},{"leaderboard":"/sota/fine-grained-image-classification-on-caltech","task":"Fine-Grained Image Classification","dataset":"Caltech-101","model":"VGG-19bn (Spinal FC)","rank_in_archive_order":7,"of":18,"metrics":{"Top-1 Error Rate":"6.84%"},"uses_additional_data":true},{"leaderboard":"/sota/fine-grained-image-classification-on-fruits","task":"Fine-Grained Image Classification","dataset":"Fruits-360","model":"VGG-19bn","rank_in_archive_order":2,"of":3,"metrics":{"Accuracy (%)":"99.90"},"uses_additional_data":true},{"leaderboard":"/sota/fine-grained-image-classification-on-oxford","task":"Fine-Grained Image Classification","dataset":"Oxford 102 Flowers","model":"Wide-ResNet-101 (Spinal FC)","rank_in_archive_order":5,"of":25,"metrics":{"Accuracy":"99.30%"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-emnist-balanced","task":"Image Classification","dataset":"EMNIST-Balanced","model":"VGG-5(Spinal FC)","rank_in_archive_order":4,"of":20,"metrics":{"Accuracy":"91.05","Trainable Parameters":"3630000"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-emnist-balanced","task":"Image Classification","dataset":"EMNIST-Balanced","model":"VGG-5","rank_in_archive_order":5,"of":20,"metrics":{"Accuracy":"91.04","Trainable Parameters":"3646000"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-emnist-balanced","task":"Image Classification","dataset":"EMNIST-Balanced","model":"CNN(Spinal FC)","rank_in_archive_order":14,"of":20,"metrics":{"Accuracy":"83.21","Trainable Parameters":"16050"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-emnist-balanced","task":"Image Classification","dataset":"EMNIST-Balanced","model":"CNN(Spinal FC)","rank_in_archive_order":15,"of":20,"metrics":{"Accuracy":"82.77","Trainable Parameters":"13820"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-emnist-balanced","task":"Image Classification","dataset":"EMNIST-Balanced","model":"CNN","rank_in_archive_order":17,"of":20,"metrics":{"Accuracy":"79.61","Trainable Parameters":"21840"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-emnist-digits","task":"Image Classification","dataset":"EMNIST-Digits","model":"VGG-5(Spinal FC)","rank_in_archive_order":3,"of":7,"metrics":{"Accuracy (%)":"99.75"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-emnist-letters","task":"Image Classification","dataset":"EMNIST-Letters","model":"VGG-5(Spinal FC)","rank_in_archive_order":4,"of":11,"metrics":{"Accuracy":"95.88"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-emnist-letters","task":"Image Classification","dataset":"EMNIST-Letters","model":"VGG-5","rank_in_archive_order":5,"of":11,"metrics":{"Accuracy":"95.86"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-flowers-102","task":"Image Classification","dataset":"Flowers-102","model":"Wide-ResNet-101 (Spinal FC)","rank_in_archive_order":13,"of":52,"metrics":{"Accuracy":"99.30"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-kuzushiji-mnist","task":"Image Classification","dataset":"Kuzushiji-MNIST","model":"VGG-5 (Spinal FC)","rank_in_archive_order":3,"of":26,"metrics":{"Accuracy":"99.15","Error":"0.85"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-mnist","task":"Image Classification","dataset":"MNIST","model":"VGG-5 (Spinal FC)","rank_in_archive_order":14,"of":81,"metrics":{"Accuracy":"99.72","Percentage error":"0.28"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-stl-10","task":"Image Classification","dataset":"STL-10","model":"Wide-ResNet-101 (Spinal FC)","rank_in_archive_order":3,"of":117,"metrics":{"Percentage correct":"98.66"},"uses_additional_data":true},{"leaderboard":"/sota/image-classification-on-stl-10","task":"Image Classification","dataset":"STL-10","model":"VGG-19bn","rank_in_archive_order":17,"of":117,"metrics":{"Percentage correct":"95.44"},"uses_additional_data":true}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}