{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dat-spatially-dynamic-vision-transformer-with","title":"DAT++: Spatially Dynamic Vision Transformer with Deformable Attention","arxiv_id":"2309.01430","date":"2023-09-04","proceeding":null,"authors":["Zhuofan Xia","Xuran Pan","Shiji Song","Li Erran Li","Gao Huang"],"abstract":"Transformers have shown superior performance on various vision tasks. Their large receptive field endows Transformer models with higher representation power than their CNN counterparts. Nevertheless, simply enlarging the receptive field also raises several concerns. On the one hand, using dense attention in ViT leads to excessive memory and computational cost, and features can be influenced by irrelevant parts that are beyond the region of interests. On the other hand, the handcrafted attention adopted in PVT or Swin Transformer is data agnostic and may limit the ability to model long-range relations. To solve this dilemma, we propose a novel deformable multi-head attention module, where the positions of key and value pairs in self-attention are adaptively allocated in a data-dependent way. This flexible scheme enables the proposed deformable attention to dynamically focus on relevant regions while maintains the representation power of global attention. On this basis, we present Deformable Attention Transformer (DAT), a general vision backbone efficient and effective for visual recognition. We further build an enhanced version DAT++. Extensive experiments show that our DAT++ achieves state-of-the-art results on various visual recognition benchmarks, with 85.9% ImageNet accuracy, 54.5 and 47.0 MS-COCO instance segmentation mAP, and 51.5 ADE20K semantic segmentation mIoU.","url_abs":"https://arxiv.org/abs/2309.01430v1","url_pdf":"https://arxiv.org/pdf/2309.01430v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dat-spatially-dynamic-vision-transformer-with","repo_url":"https://github.com/leaplabthu/dat","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"image-classification","task_name":"Image Classification"},{"task_slug":"instance-segmentation","task_name":"Instance Segmentation"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"semantic-segmentation","task_name":"Semantic Segmentation"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"focus","method_name":"Focus"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"pvt","method_name":"PVT"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"spatial-reduction-attention","method_name":"Spatial-Reduction Attention"},{"method_slug":"stochastic-depth","method_name":"Stochastic Depth"},{"method_slug":"swin-transformer","method_name":"Swin Transformer"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"DAT-B++ (384x384)","rank_in_archive_order":187,"of":1060,"metrics":{"GFLOPs":"49.7","Number of params":"94M","Top 1 Accuracy":"85.9%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"DAT-B++ (224x224)","rank_in_archive_order":280,"of":1060,"metrics":{"GFLOPs":"16.6","Number of params":"93M","Top 1 Accuracy":"84.9%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"DAT-S++","rank_in_archive_order":302,"of":1060,"metrics":{"GFLOPs":"9.4","Number of params":"53M","Top 1 Accuracy":"84.6%"},"uses_additional_data":false},{"leaderboard":"/sota/image-classification-on-imagenet","task":"Image Classification","dataset":"ImageNet","model":"DAT-T++","rank_in_archive_order":377,"of":1060,"metrics":{"GFLOPs":"4.3","Number of params":"24M","Top 1 Accuracy":"83.9%"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-coco-2017","task":"Object Detection","dataset":"COCO 2017","model":"DAT-S++","rank_in_archive_order":4,"of":24,"metrics":{"AP":"50.2"},"uses_additional_data":false},{"leaderboard":"/sota/object-detection-on-coco-2017","task":"Object Detection","dataset":"COCO 2017","model":"DAT-T++","rank_in_archive_order":5,"of":24,"metrics":{"AP":"49.2"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-ade20k","task":"Semantic Segmentation","dataset":"ADE20K","model":"DAT-B++","rank_in_archive_order":96,"of":235,"metrics":{"Validation mIoU":"51.5"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-ade20k","task":"Semantic Segmentation","dataset":"ADE20K","model":"DAT-S++","rank_in_archive_order":99,"of":235,"metrics":{"Validation mIoU":"51.2"},"uses_additional_data":false},{"leaderboard":"/sota/semantic-segmentation-on-ade20k","task":"Semantic Segmentation","dataset":"ADE20K","model":"DAT-T++","rank_in_archive_order":115,"of":235,"metrics":{"Validation mIoU":"50.3"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2309.01430","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2309.01430"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/leaplabthu/dat","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_honours":1,"ran":1,"unverified":8},"by_repo_kind":{"official":{"samples":10,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"eec1e7cba51d5e8e","entry":"get_grad_norm","repo":"leaplabthu/dat","repo_kind":"official","path":"utils.py","file_url":"https://github.com/leaplabthu/dat/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"eec1e7cba51d5e8e"}},{"code_sha256_prefix":"7ee38b80fc4e6b88","entry":"load_checkpoint","repo":"leaplabthu/dat","repo_kind":"official","path":"utils.py","file_url":"https://github.com/leaplabthu/dat/blob/HEAD/utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7ee38b80fc4e6b88"}},{"code_sha256_prefix":"0c32459bef92ff8c","entry":"auto_resume_helper","repo":"leaplabthu/dat","repo_kind":"official","path":"utils.py","file_url":"https://github.com/leaplabthu/dat/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0c32459bef92ff8c"}},{"code_sha256_prefix":"376b1ee5cd01affe","entry":"build_model","repo":"leaplabthu/dat","repo_kind":"official","path":"models/build.py","file_url":"https://github.com/leaplabthu/dat/blob/HEAD/models/build.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"376b1ee5cd01affe"}},{"code_sha256_prefix":"fcf2d41dd445a70b","entry":"build_optimizer","repo":"leaplabthu/dat","repo_kind":"official","path":"optimizer.py","file_url":"https://github.com/leaplabthu/dat/blob/HEAD/optimizer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fcf2d41dd445a70b"}},{"code_sha256_prefix":"07e8b4b487c0abb6","entry":"build_scheduler","repo":"leaplabthu/dat","repo_kind":"official","path":"lr_scheduler.py","file_url":"https://github.com/leaplabthu/dat/blob/HEAD/lr_scheduler.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"07e8b4b487c0abb6"}},{"code_sha256_prefix":"b0a5beb34716d5a6","entry":"check_keywords_in_name","repo":"leaplabthu/dat","repo_kind":"official","path":"optimizer.py","file_url":"https://github.com/leaplabthu/dat/blob/HEAD/optimizer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b0a5beb34716d5a6"}},{"code_sha256_prefix":"82e52feafbf040bb","entry":"create_logger","repo":"leaplabthu/dat","repo_kind":"official","path":"logger.py","file_url":"https://github.com/leaplabthu/dat/blob/HEAD/logger.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"82e52feafbf040bb"}},{"code_sha256_prefix":"5a38783eb3898fcc","entry":"get_config","repo":"leaplabthu/dat","repo_kind":"official","path":"config.py","file_url":"https://github.com/leaplabthu/dat/blob/HEAD/config.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5a38783eb3898fcc"}},{"code_sha256_prefix":"104af48a94364f24","entry":"set_weight_decay_and_lr","repo":"leaplabthu/dat","repo_kind":"official","path":"optimizer.py","file_url":"https://github.com/leaplabthu/dat/blob/HEAD/optimizer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"104af48a94364f24"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}