{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/gmtr-graph-matching-transformers","title":"GMTR: Graph Matching Transformers","arxiv_id":"2311.08141","date":"2023-11-14","proceeding":null,"authors":["Jinpei Guo","Shaofeng Zhang","Runzhong Wang","Chang Liu","Junchi Yan"],"abstract":"Vision transformers (ViTs) have recently been used for visual matching beyond object detection and segmentation. However, the original grid dividing strategy of ViTs neglects the spatial information of the keypoints, limiting the sensitivity to local information. Therefore, we propose QueryTrans (Query Transformer), which adopts a cross-attention module and keypoints-based center crop strategy for better spatial information extraction. We further integrate the graph attention module and devise a transformer-based graph matching approach GMTR (Graph Matching TRansformers) whereby the combinatorial nature of GM is addressed by a graph transformer neural GM solver. On standard GM benchmarks, GMTR shows competitive performance against the SOTA frameworks. Specifically, on Pascal VOC, GMTR achieves $\\mathbf{83.6\\%}$ accuracy, $\\mathbf{0.9\\%}$ higher than the SOTA framework. On Spair-71k, GMTR shows great potential and outperforms most of the previous works. Meanwhile, on Pascal VOC, QueryTrans improves the accuracy of NGMv2 from $80.1\\%$ to $\\mathbf{83.3\\%}$, and BBGM from $79.0\\%$ to $\\mathbf{84.5\\%}$. On Spair-71k, QueryTrans improves NGMv2 from $80.6\\%$ to $\\mathbf{82.5\\%}$, and BBGM from $82.1\\%$ to $\\mathbf{83.9\\%}$. Source code will be made publicly available.","url_abs":"https://arxiv.org/abs/2311.08141v2","url_pdf":"https://arxiv.org/pdf/2311.08141v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"gmtr-graph-matching-transformers","repo_url":"https://github.com/jp-guo/gm-transformer","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"graph-attention","task_name":"Graph Attention"},{"task_slug":"graph-matching","task_name":"Graph Matching"},{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"concatenated-skip-connection","method_name":"Concatenated Skip Connection"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"graph-transformer","method_name":"Graph Transformer"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"lapeigen","method_name":"LapEigen"},{"method_slug":"laplacian-pe","method_name":"Laplacian PE"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/graph-matching-on-pascal-voc","task":"Graph Matching","dataset":"PASCAL VOC","model":"GMT-BBGM","rank_in_archive_order":13,"of":31,"metrics":{"matching accuracy":"0.8411"},"uses_additional_data":false},{"leaderboard":"/sota/graph-matching-on-pascal-voc","task":"Graph Matching","dataset":"PASCAL VOC","model":"GMTR","rank_in_archive_order":14,"of":31,"metrics":{"matching accuracy":"0.836"},"uses_additional_data":false},{"leaderboard":"/sota/graph-matching-on-spair-71k","task":"Graph Matching","dataset":"SPair-71k","model":"GMTR","rank_in_archive_order":3,"of":8,"metrics":{"matching accuracy":"0.832"},"uses_additional_data":false},{"leaderboard":"/sota/graph-matching-on-spair-71k","task":"Graph Matching","dataset":"SPair-71k","model":"GMT-BBGM","rank_in_archive_order":4,"of":8,"metrics":{"matching accuracy":"0.8296"},"uses_additional_data":false},{"leaderboard":"/sota/graph-matching-on-willow-object-class","task":"Graph Matching","dataset":"Willow Object Class","model":"GMT-BBGM","rank_in_archive_order":6,"of":23,"metrics":{"matching accuracy":"0.9813"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}