{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-multimodal-subspace-clustering-networks","title":"Deep Multimodal Subspace Clustering Networks","arxiv_id":"1804.06498","date":"2018-04-17","proceeding":null,"authors":["Mahdi Abavisani","Vishal M. Patel"],"abstract":"We present convolutional neural network (CNN) based approaches for\nunsupervised multimodal subspace clustering. The proposed framework consists of\nthree main stages - multimodal encoder, self-expressive layer, and multimodal\ndecoder. The encoder takes multimodal data as input and fuses them to a latent\nspace representation. The self-expressive layer is responsible for enforcing\nthe self-expressiveness property and acquiring an affinity matrix corresponding\nto the data points. The decoder reconstructs the original input data. The\nnetwork uses the distance between the decoder's reconstruction and the original\ninput in its training. We investigate early, late and intermediate fusion\ntechniques and propose three different encoders corresponding to them for\nspatial fusion. The self-expressive layers and multimodal decoders are\nessentially the same for different spatial fusion-based approaches. In addition\nto various spatial fusion-based methods, an affinity fusion-based network is\nalso proposed in which the self-expressive layer corresponding to different\nmodalities is enforced to be the same. Extensive experiments on three datasets\nshow that the proposed methods significantly outperform the state-of-the-art\nmultimodal subspace clustering methods.","url_abs":"http://arxiv.org/abs/1804.06498v3","url_pdf":"http://arxiv.org/pdf/1804.06498v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-multimodal-subspace-clustering-networks","repo_url":"https://github.com/mahdiabavisani/Deep-multimodal-subspace-clustering-networks","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok"}}],"tasks":[{"task_slug":"clustering","task_name":"Clustering"},{"task_slug":"decoder","task_name":"Decoder"},{"task_slug":"image-clustering","task_name":"Image Clustering"},{"task_slug":"multi-modal-subspace-clustering","task_name":"Multi-modal Subspace Clustering"},{"task_slug":"multi-view-subspace-clustering","task_name":"Multi-view Subspace Clustering"},{"task_slug":"multiview-learning","task_name":"Multiview Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/image-clustering-on-arl-polarimetric-thermal","task":"Image Clustering","dataset":"ARL Polarimetric Thermal Face Dataset","model":"DMSC","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy":"0.983"},"uses_additional_data":false},{"leaderboard":"/sota/image-clustering-on-extended-yale-b","task":"Image Clustering","dataset":"Extended Yale-B","model":"DMSC","rank_in_archive_order":1,"of":9,"metrics":{"Accuracy":"0.992","NMI":"0.988"},"uses_additional_data":false},{"leaderboard":"/sota/image-clustering-on-usps","task":"Image Clustering","dataset":"USPS","model":"DMSC","rank_in_archive_order":7,"of":16,"metrics":{"Accuracy":"0.951","NMI":"0.929"},"uses_additional_data":true},{"leaderboard":"/sota/multi-view-subspace-clustering-on-arl","task":"Multi-view Subspace Clustering","dataset":"ARL Polarimetric Thermal Face Dataset","model":"DMSC","rank_in_archive_order":1,"of":1,"metrics":{"Accuracy":"0.988"},"uses_additional_data":false},{"leaderboard":"/sota/multi-view-subspace-clustering-on-orl","task":"Multi-view Subspace Clustering","dataset":"ORL","model":"DMSC","rank_in_archive_order":2,"of":3,"metrics":{"Accuracy":"0.833"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/1804.06498","atlas_url":"https://app.syntology.ai/?focus=1804.06498","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}