Papers › Learning Robust Visual-Semantic Embeddings
Learning Robust Visual-Semantic Embeddings
Yao-Hung Hubert Tsai, Liang-Kang Huang, Ruslan Salakhutdinov
Many of the existing methods for learning joint embedding of images and text use only supervised information from paired images and its textual attributes. Taking advantage of the recent success of unsupervised learning in deep neural networks, we propose an end-to-end learning framework that is able to extract more robust multi-modal representations across domains. The proposed method combines representation learning models (i.e., auto-encoders) together with cross-domain learning criteria (i.e., Maximum Mean Discrepancy loss) to learn joint embeddings for semantic and visual features. A novel technique of unsupervised-data adaptation inference is introduced to construct more comprehensive embeddings for both labeled and unlabeled data. We evaluate our method on Animals with Attributes and Caltech-UCSD Birds 200-2011 dataset with a wide range of applications, including zero and few-shot image recognition and retrieval, from inductive to transductive settings. Empirically, we show that our framework improves over the current state of the art on many of the considered tasks.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
No code repository is listed for this paper in the archive or in Syntology's graph.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Generalized Few-Shot Learning | AwA2 | REVISE | Per-Class Accuracy (1-shot) | 56.1 | #6 of 6 | Archive leaderboard | report |
| Generalized Few-Shot Learning | AwA2 | REVISE | Per-Class Accuracy (10-shots) | 67.8 | #6 of 6 | Archive leaderboard | report |
| Generalized Few-Shot Learning | AwA2 | REVISE | Per-Class Accuracy (2-shots) | 60.3 | #6 of 6 | Archive leaderboard | report |
| Generalized Few-Shot Learning | AwA2 | REVISE | Per-Class Accuracy (5-shots) | 64.1 | #6 of 6 | Archive leaderboard | report |
| Generalized Few-Shot Learning | CUB | REVISE | Per-Class Accuracy (2-shots) | 41.1 | #5 of 5 | Archive leaderboard | report |
| Generalized Few-Shot Learning | CUB | REVISE | Per-Class Accuracy (1-shot) | 36.3 | #5 of 5 | Archive leaderboard | report |
| Generalized Few-Shot Learning | CUB | REVISE | Per-Class Accuracy (10-shots) | 50.9 | #5 of 5 | Archive leaderboard | report |
| Generalized Few-Shot Learning | CUB | REVISE | Per-Class Accuracy (5-shots) | 44.6 | #5 of 5 | Archive leaderboard | report |
| Generalized Few-Shot Learning | SUN | REVISE | Per-Class Accuracy (1-shot) | 27.4 | #5 of 5 | Archive leaderboard | report |
| Generalized Few-Shot Learning | SUN | REVISE | Per-Class Accuracy (10-shots) | 40.8 | #5 of 5 | Archive leaderboard | report |
| Generalized Few-Shot Learning | SUN | REVISE | Per-Class Accuracy (2-shots) | 33.4 | #5 of 5 | Archive leaderboard | report |
| Generalized Few-Shot Learning | SUN | REVISE | Per-Class Accuracy (5-shots) | 37.4 | #5 of 5 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections