{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/multi-modal-cycle-consistent-generalized-zero","title":"Multi-modal Cycle-consistent Generalized Zero-Shot Learning","arxiv_id":"1808.00136","date":"2018-08-01","proceeding":"ECCV 2018 9","authors":["Rafael Felix","B. G. Vijay Kumar","Ian Reid","Gustavo Carneiro"],"abstract":"In generalized zero shot learning (GZSL), the set of classes are split into\nseen and unseen classes, where training relies on the semantic features of the\nseen and unseen classes and the visual representations of only the seen\nclasses, while testing uses the visual representations of the seen and unseen\nclasses. Current methods address GZSL by learning a transformation from the\nvisual to the semantic space, exploring the assumption that the distribution of\nclasses in the semantic and visual spaces is relatively similar. Such methods\ntend to transform unseen testing visual representations into one of the seen\nclasses' semantic features instead of the semantic features of the correct\nunseen class, resulting in low accuracy GZSL classification. Recently,\ngenerative adversarial networks (GAN) have been explored to synthesize visual\nrepresentations of the unseen classes from their semantic features - the\nsynthesized representations of the seen and unseen classes are then used to\ntrain the GZSL classifier. This approach has been shown to boost GZSL\nclassification accuracy, however, there is no guarantee that synthetic visual\nrepresentations can generate back their semantic feature in a multi-modal\ncycle-consistent manner. This constraint can result in synthetic visual\nrepresentations that do not represent well their semantic features. In this\npaper, we propose the use of such constraint based on a new regularization for\nthe GAN training that forces the generated visual features to reconstruct their\noriginal semantic features. Once our model is trained with this multi-modal\ncycle-consistent semantic compatibility, we can then synthesize more\nrepresentative visual representations for the seen and, more importantly, for\nthe unseen classes. Our proposed approach shows the best GZSL classification\nresults in the field in several publicly available datasets.","url_abs":"http://arxiv.org/abs/1808.00136v2","url_pdf":"http://arxiv.org/pdf/1808.00136v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"multi-modal-cycle-consistent-generalized-zero","repo_url":"https://github.com/rfelixmg/frwgan-eccv18","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok"}}],"tasks":[{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"generalized-zero-shot-learning","task_name":"Generalized Zero-Shot Learning"},{"task_slug":"zero-shot-learning","task_name":"Zero-Shot Learning"}],"methods":[{"method_slug":"convolution","method_name":"Convolution"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/generalized-zero-shot-learning-on-sun","task":"Generalized Zero-Shot Learning","dataset":"SUN Attribute","model":"Cycle-WGAN","rank_in_archive_order":6,"of":9,"metrics":{"Harmonic mean":"39.4"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-learning-on-cub-200-2011","task":"Zero-Shot Learning","dataset":"CUB-200-2011","model":"Cycle-WGAN","rank_in_archive_order":11,"of":14,"metrics":{"average top-1 classification accuracy":"58.6"},"uses_additional_data":false},{"leaderboard":"/sota/zero-shot-learning-on-sun-attribute","task":"Zero-Shot Learning","dataset":"SUN Attribute","model":"Cycle-WGAN","rank_in_archive_order":9,"of":9,"metrics":{"average top-1 classification accuracy":"59.9"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1808.00136","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}