Papers › Bridge Correlational Neural Networks for Multilingual Multimodal Representation Learning
Bridge Correlational Neural Networks for Multilingual Multimodal Representation Learning
Janarthanan Rajendran, Mitesh M. Khapra, Sarath Chandar, Balaraman Ravindran
Recently there has been a lot of interest in learning common representations for multiple views of data. Typically, such common representations are learned using a parallel corpus between the two views (say, 1M images and their English captions). In this work, we address a real-world scenario where no direct parallel data is available between two views of interest (say, V₁ and V₂) but parallel data is available between each of these views and a pivot view (V₃). We propose a model for learning a common representation for V₁, V₂ and V₃ using only the parallel data available between V₁V₃ and V₂V₃. The proposed model is generic and even works when there are n views of interest and only one pivot view which acts as a bridge between them. There are two specific downstream applications that we focus on (i) transfer learning between languages L₁,L₂,...,Lₙ using a pivot language L and (ii) cross modal access between images and a language L₁ using a pivot language L₂. Our model achieves state-of-the-art performance in multilingual document classification on the publicly available multilingual TED corpus and promising results in multilingual multimodal retrieval on a new dataset created and released as a part of this work.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
No leaderboard rows for this paper in the archive.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections