Papers › VICTR: Visual Information Captured Text Representation for Text-to-Image Multimodal Tasks
VICTR: Visual Information Captured Text Representation for Text-to-Image Multimodal Tasks
Soyeon Caren Han, Siqu Long, Siwen Luo, Kunze Wang, Josiah Poon
Text-to-image multimodal tasks, generating/retrieving an image from a given text description, are extremely challenging tasks since raw text descriptions cover quite limited information in order to fully describe visually realistic images. We propose a new visual contextual text representation for text-to-image multimodal tasks, VICTR, which captures rich visual semantic information of objects from the text input. First, we use the text description as initial input and conduct dependency parsing to extract the syntactic structure and analyse the semantic aspect, including object quantities, to extract the scene graph. Then, we train the extracted objects, attributes, and relations in the scene graph and the corresponding geometric relation information using Graph Convolutional Networks, and it generates text representation which integrates textual and visual semantic information. The text representation is aggregated with word-level and sentence-level embedding to generate both visual contextual word and sentence representation. For the evaluation, we attached VICTR to the state-of-the-art models in text-to-image generation.VICTR is easily added to existing models and improves across both quantitative and qualitative aspects.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Text-to-Image Generation | COCO (Common Objects in Context) | AttnGAN + VICTR | FID | 29.26 | #57 of 69 | Archive leaderboard | report |
| Text-to-Image Generation | COCO (Common Objects in Context) | AttnGAN + VICTR | Inception score | 28.18 | #57 of 69 | Archive leaderboard | report |
| Text-to-Image Generation | COCO (Common Objects in Context) | DM-GAN + VICTR | FID | 32.37 | #59 of 69 | Archive leaderboard | report |
| Text-to-Image Generation | COCO (Common Objects in Context) | DM-GAN + VICTR | Inception score | 32.37 | #59 of 69 | Archive leaderboard | report |
| Text-to-Image Generation | COCO (Common Objects in Context) | StackGAN + VICTR | Inception score | 10.38 | #68 of 69 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections