{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ts-net-combining-modality-specific-and-common","title":"TS-Net: Combining modality specific and common features for multimodal patch matching","arxiv_id":"1806.01550","date":"2018-06-05","proceeding":null,"authors":["Sovann En","Alexis Lechervy","Frédéric Jurie"],"abstract":"Multimodal patch matching addresses the problem of finding the\ncorrespondences between image patches from two different modalities, e.g. RGB\nvs sketch or RGB vs near-infrared. The comparison of patches of different\nmodalities can be done by discovering the information common to both modalities\n(Siamese like approaches) or the modality-specific information (Pseudo-Siamese\nlike approaches). We observed that none of these two scenarios is optimal. This\nmotivates us to propose a three-stream architecture, dubbed as TS-Net,\ncombining the benefits of the two. In addition, we show that adding extra\nconstraints in the intermediate layers of such networks further boosts the\nperformance. Experimentations on three multimodal datasets show significant\nperformance gains in comparison with Siamese and Pseudo-Siamese networks.","url_abs":"http://arxiv.org/abs/1806.01550v1","url_pdf":"http://arxiv.org/pdf/1806.01550v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ts-net-combining-modality-specific-and-common","repo_url":"https://github.com/ensv/TS-Net","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":null}],"tasks":[{"task_slug":"multimodal-patch-matching","task_name":"Multimodal Patch Matching"},{"task_slug":"patch-matching","task_name":"Patch Matching"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}