{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/spinenet-learning-scale-permuted-backbone-for","title":"SpineNet: Learning Scale-Permuted Backbone for Recognition and Localization","arxiv_id":"1912.05027","date":"2019-12-10","proceeding":"CVPR 2020 6","authors":["Xianzhi Du","Tsung-Yi Lin","Pengchong Jin","Golnaz Ghiasi","Mingxing Tan","Yin Cui","Quoc V. Le","Xiaodan Song"],"abstract":"Convolutional neural networks typically encode an input image into a series of intermediate features with decreasing resolutions. While this structure is suited to classification tasks, it does not perform well for tasks requiring simultaneous recognition and localization (e.g., object detection). The encoder-decoder architectures are proposed to resolve this by applying a decoder network onto a backbone model designed for classification tasks. In this paper, we argue encoder-decoder architecture is ineffective in generating strong multi-scale features because of the scale-decreased backbone. We propose SpineNet, a backbone with scale-permuted intermediate features and cross-scale connections that is learned on an object detection task by Neural Architecture Search. Using similar building blocks, SpineNet models outperform ResNet-FPN models by ~3% AP at various scales while using 10-20% fewer FLOPs. In particular, SpineNet-190 achieves 52.5% AP with a MaskR-CNN detector and achieves 52.1% AP with a RetinaNet detector on COCO for a single model without test-time augmentation, significantly outperforms prior art of detectors. SpineNet can transfer to classification tasks, achieving 5% top-1 accuracy improvement on a challenging iNaturalist fine-grained dataset. Code is at: https://github.com/tensorflow/tpu/tree/master/models/official/detection.","url_abs":"https://arxiv.org/abs/1912.05027v3","url_pdf":"https://arxiv.org/pdf/1912.05027v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"spinenet-learning-scale-permuted-backbone-for","repo_url":"https://github.com/tensorflow/tpu","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"spinenet-learning-scale-permuted-backbone-for","repo_url":"https://github.com/tensorflow/models/tree/master/official/vision/beta/modeling/backbones","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":null},{"paper_slug":"spinenet-learning-scale-permuted-backbone-for","repo_url":"https://github.com/lucifer443/SpineNet-Pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"spinenet-learning-scale-permuted-backbone-for","repo_url":"https://github.com/tensorflow/tpu/tree/master/models/official/detection","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"spinenet-learning-scale-permuted-backbone-for","repo_url":"https://github.com/yan-roo/SpineNet-Pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"spinenet-learning-scale-permuted-backbone-for","repo_url":"https://github.com/2023-MindSpore-4/Code11/tree/main/retinanet_resnet152","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"spinenet-learning-scale-permuted-backbone-for","repo_url":"https://github.com/2023-MindSpore-4/Code14/tree/main/SpineNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"spinenet-learning-scale-permuted-backbone-for","repo_url":"https://github.com/2023-MindSpore-4/Code14/tree/main/retinanet_resnet101","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"spinenet-learning-scale-permuted-backbone-for","repo_url":"https://github.com/2023-MindSpore-4/Code14/tree/main/retinanet_resnet152","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"spinenet-learning-scale-permuted-backbone-for","repo_url":"https://github.com/2023-MindSpore-4/Code9/tree/main/RetinaNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"spinenet-learning-scale-permuted-backbone-for","repo_url":"https://github.com/MindSpore-paper-code-2/code3/tree/main/ssd_resnet_34","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"spinenet-learning-scale-permuted-backbone-for","repo_url":"https://github.com/code-implementation1/Code7/tree/main/retinanet_resnet101","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"spinenet-learning-scale-permuted-backbone-for","repo_url":"https://github.com/code-implementation1/Code8/tree/main/SpineNet","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"instance-segmentation","task_name":"Instance Segmentation"},{"task_slug":"architecture-search","task_name":"Neural Architecture Search"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"point-cloud-registration","task_name":"Point Cloud Registration"},{"task_slug":"real-time-object-detection","task_name":"Real-Time Object Detection"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"cosine-annealing","method_name":"Cosine Annealing"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"entropy-regularization","method_name":"Entropy Regularization"},{"method_slug":"fpn","method_name":"FPN"},{"method_slug":"focal-loss","method_name":"Focal Loss"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"linear-warmup-with-cosine-annealing","method_name":"Linear Warmup With Cosine Annealing"},{"method_slug":"mask-r-cnn","method_name":"Mask R-CNN"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"nas-fpn","method_name":"NAS-FPN"},{"method_slug":"neural-architecture-search","method_name":"Neural Architecture Search"},{"method_slug":"ppo","method_name":"PPO"},{"method_slug":"rpn","method_name":"RPN"},{"method_slug":"randomhorizontalflip","method_name":"Random Horizontal Flip"},{"method_slug":"random-resized-crop","method_name":"Random Resized Crop"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"retinanet","method_name":"RetinaNet"},{"method_slug":"roi-align","method_name":"RoIAlign"},{"method_slug":"sgd-with-momentum","method_name":"SGD with Momentum"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"spinenet","method_name":"SpineNet"},{"method_slug":"stochastic-depth","method_name":"Stochastic Depth"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"},{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[{"slug":"spinenet","name":"SpineNet","full_name":"SpineNet"}],"results":[{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"SpineNet-143","rank_in_archive_order":794,"of":1060,"metrics":{"GFLOPs":"9.1","Number of params":"60.5M","Top 1 Accuracy":"79%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-inaturalist","task":"Image Classification","dataset":"iNaturalist","model":"SpineNet-143","rank_in_archive_order":15,"of":19,"metrics":{"Top 1 Accuracy":"63.6%","Top 5 Accuracy":"84.8%"},"uses_additional_data":false},{"leaderboard":"/sota/instance-segmentation-on-coco-minival","task":"Instance Segmentation","dataset":"COCO minival","model":"RetinaNet (SpineNet-190, 1536x1536)","rank_in_archive_order":45,"of":93,"metrics":{"mask AP":"46.1"},"uses_additional_data":false},{"leaderboard":"/sota/instance-segmentation-on-coco","task":"Instance Segmentation","dataset":"COCO test-dev","model":"Mask R-CNN (SpineNet-190, 1536x1536)","rank_in_archive_order":38,"of":112,"metrics":{"mask AP":"46.1"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-coco-minival","task":"Object Detection","dataset":"COCO minival","model":"RetinaNet (SpineNet-190, 1536x1536)","rank_in_archive_order":67,"of":220,"metrics":{"box AP":"52.2"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-coco","task":"Object Detection","dataset":"COCO test-dev","model":"RetinaNet (SpineNet-190, 1280x1280)","rank_in_archive_order":74,"of":225,"metrics":{"AP50":"71.8","AP75":"56.5","APL":"63.6","APM":"55","APS":"35.4","box mAP":"52.1"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-coco","task":"Object Detection","dataset":"COCO test-dev","model":"RetinaNet (SpineNet-143, 1280x1280)","rank_in_archive_order":85,"of":225,"metrics":{"AP50":"70.4","AP75":"54.9","APL":"62.1","APM":"53.9","APS":"33.6","box mAP":"50.7"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-coco","task":"Object Detection","dataset":"COCO test-dev","model":"RetinaNet (SpineNet-96, 1024x1024)","rank_in_archive_order":104,"of":225,"metrics":{"AP50":"68.4","AP75":"52.5","APL":"62","APM":"52.3","APS":"32","box mAP":"48.6"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-coco","task":"Object Detection","dataset":"COCO test-dev","model":"RetinaNet (SpineNet-49, 896x896)","rank_in_archive_order":124,"of":225,"metrics":{"AP50":"66.3","AP75":"50.6","APL":"61.7","APM":"50.1","APS":"29.1","box mAP":"46.7"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-coco","task":"Object Detection","dataset":"COCO test-dev","model":"RetinaNet (SpineNet-49, 640x640)","rank_in_archive_order":146,"of":225,"metrics":{"AP50":"63.8","AP75":"47.6","APL":"61.1","APM":"47.7","APS":"25.9","box mAP":"44.3"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-coco","task":"Object Detection","dataset":"COCO test-dev","model":"SpineNet-49 (640, RetinaNet, single-scale)","rank_in_archive_order":167,"of":225,"metrics":{"AP50":"62.3","AP75":"46.1","APL":"57.3","APM":"45.2","APS":"23.7","box mAP":"42.8"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-coco","task":"Object Detection","dataset":"COCO test-dev","model":"RetinaNet (SpineNet-49S, 640x640)","rank_in_archive_order":183,"of":225,"metrics":{"AP50":"60.5","AP75":"44.6","APL":"58","APM":"45","APS":"23.3","box mAP":"41.5"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1912.05027","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}