{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unsupervised-cross-spectral-stereo-matching","title":"Unsupervised Cross-spectral Stereo Matching by Learning to Synthesize","arxiv_id":"1903.01078","date":"2019-03-04","proceeding":null,"authors":["Mingyang Liang","Xiaoyang Guo","Hongsheng Li","Xiaogang Wang","You Song"],"abstract":"Unsupervised cross-spectral stereo matching aims at recovering disparity\ngiven cross-spectral image pairs without any supervision in the form of ground\ntruth disparity or depth. The estimated depth provides additional information\ncomplementary to individual semantic features, which can be helpful for other\nvision tasks such as tracking, recognition and detection. However, there are\nlarge appearance variations between images from different spectral bands, which\nis a challenge for cross-spectral stereo matching. Existing deep unsupervised\nstereo matching methods are sensitive to the appearance variations and do not\nperform well on cross-spectral data. We propose a novel unsupervised\ncross-spectral stereo matching framework based on image-to-image translation.\nFirst, a style adaptation network transforms images across different spectral\nbands by cycle consistency and adversarial learning, during which appearance\nvariations are minimized. Then, a stereo matching network is trained with image\npairs from the same spectra using view reconstruction loss. At last, the\nestimated disparity is utilized to supervise the spectral-translation network\nin an end-to-end way. Moreover, a novel style adaptation network F-cycleGAN is\nproposed to improve the robustness of spectral translation. Our method can\ntackle appearance variations and enhance the robustness of unsupervised\ncross-spectral stereo matching. Experimental results show that our method\nachieves good performance without using depth supervision or explicit semantic\ninformation.","url_abs":"http://arxiv.org/abs/1903.01078v1","url_pdf":"http://arxiv.org/pdf/1903.01078v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unsupervised-cross-spectral-stereo-matching","repo_url":"https://github.com/rish-av/gan_spectral_matching","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"image-to-image-translation","task_name":"Image-to-Image Translation"},{"task_slug":"stereo-matching-1","task_name":"Stereo Matching"},{"task_slug":"stereo-matching","task_name":"Stereo Matching Hand"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1903.01078","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}