{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/connecting-look-and-feel-associating-the","title":"Connecting Look and Feel: Associating the visual and tactile properties of physical materials","arxiv_id":"1704.03822","date":"2017-04-12","proceeding":"CVPR 2017 7","authors":["Wenzhen Yuan","Shaoxiong Wang","Siyuan Dong","Edward Adelson"],"abstract":"For machines to interact with the physical world, they must understand the\nphysical properties of objects and materials they encounter. We use fabrics as\nan example of a deformable material with a rich set of mechanical properties. A\nthin flexible fabric, when draped, tends to look different from a heavy stiff\nfabric. It also feels different when touched. Using a collection of 118 fabric\nsample, we captured color and depth images of draped fabrics along with tactile\ndata from a high resolution touch sensor. We then sought to associate the\ninformation from vision and touch by jointly training CNNs across the three\nmodalities. Through the CNN, each input, regardless of the modality, generates\nan embedding vector that records the fabric's physical property. By comparing\nthe embeddings, our system is able to look at a fabric image and predict how it\nwill feel, and vice versa. We also show that a system jointly trained on vision\nand touch data can outperform a similar system trained only on visual data when\ntested purely with visual inputs.","url_abs":"http://arxiv.org/abs/1704.03822v1","url_pdf":"http://arxiv.org/pdf/1704.03822v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"connecting-look-and-feel-associating-the","repo_url":"https://github.com/wx405557858/FabricGel","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1704.03822","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}