cross-modal alignment

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

1 dataset whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 151 papers with code (342 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking
  18 Apr 2022 4 repositories listed
  In this paper, we propose \textbf{LayoutLMv3} to pre-train multimodal Transformers for Document AI with unified text and image masking.
• mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections
  24 May 2022 3 repositories listed
  Large-scale pretrained foundation models have been an emerging paradigm for building artificial intelligence (AI) systems, which can be quickly adapted to a wide range of downstream tasks.
• Skywork-R1V3 Technical Report
  8 Jul 2025 2 repositories listed Syntology ran 3 of 12 samples · 9 unverified · 12 pointer-only (licence)
  The strong performance of Skywork-R1V3 primarily stems from our elaborate post-training RL framework, which effectively activates and enhances the model's reasoning ability, without the need for additional continue…
• DeSTA2.5-Audio: Toward General-Purpose Large Audio Language Model with Self-Generated Cross-Modal Alignment
  3 Jul 2025 2 repositories listed Syntology ran 0 of 1 samples · 1 unverified · 1 pointer-only (licence)
  We introduce DeSTA2.
• Multi-Granularity Cross-modal Alignment for Generalized Medical Visual Representation Learning
  12 Oct 2022 2 repositories listed Syntology ran 0 of 1 samples · 1 unverified · 1 pointer-only (licence)
  In this paper, we present a novel Multi-Granularity Cross-modal Alignment (MGCA) framework for generalized medical visual representation learning by harnessing the naturally exhibited semantic correspondences between…
• BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learning
  17 Jun 2022 2 repositories listed Syntology ran 0 of 1 samples · 1 unverified
  Vision-Language (VL) models with the Two-Tower architecture have dominated visual-language representation learning in recent years.
• RSRefSeg 2: Decoupling Referring Remote Sensing Image Segmentation with Foundation Models
  8 Jul 2025 1 repository listed
  Referring Remote Sensing Image Segmentation provides a flexible and fine-grained framework for remote sensing scene analysis via vision-language collaborative interpretation.
• Flash-VStream: Efficient Real-Time Understanding for Long Video Streams
  30 Jun 2025 1 repository listed Syntology ran 7 of 11 samples · 4 unverified
  Benefiting from the advances in large language models and cross-modal alignment, existing multimodal large language models have achieved prominent performance in image and short video understanding.
• HyperPath: Knowledge-Guided Hyperbolic Semantic Hierarchy Modeling for WSI Analysis
  19 Jun 2025 1 repository listed
  Pathology is essential for cancer diagnosis, with multiple instance learning (MIL) widely used for whole slide image (WSI) analysis.
• ReID5o: Achieving Omni Multi-modal Person Re-identification in a Single Model
  11 Jun 2025 1 repository listed
  In real-word scenarios, person re-identification (ReID) expects to identify a person-of-interest via the descriptive query, regardless of whether the query is a single modality or a combination of multiple modalities.
• OmniDRCA: Parallel Speech-Text Foundation Model via Dual-Resolution Speech Representations and Contrastive Alignment
  11 Jun 2025 1 repository listed
  This paper presents OmniDRCA, a parallel speech-text foundation model based on joint autoregressive modeling, featuring dual-resolution speech representations and contrastive cross-modal alignment.
• Modality Curation: Building Universal Embeddings for Advanced Multimodal Information Retrieval
  26 May 2025 1 repository listed Syntology ran 3 of 9 samples · 6 unverified
  Multimodal information retrieval (MIR) faces inherent challenges due to the heterogeneity of data sources and the complexity of cross-modal alignment.
• ICPL-ReID: Identity-Conditional Prompt Learning for Multi-Spectral Object Re-Identification
  23 May 2025 1 repository listed
  Specifically, we first propose the online prompt learning using learnable text prompt as the identity-level semantic center to bridge the identity semantics of different spectra in online manner.
• U-SAM: An audio language Model for Unified Speech, Audio, and Music Understanding
  20 May 2025 1 repository listed
  The text generation paradigm for audio tasks has opened new possibilities for unified audio understanding.
• MathCoder-VL: Bridging Vision and Code for Enhanced Multimodal Mathematical Reasoning
  15 May 2025 1 repository listed Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
  Natural language image-caption datasets, widely used for training Large Multimodal Models, mainly focus on natural scenarios and overlook the intricate details of mathematical figures that are critical for…
• MSCI: Addressing CLIP's Inherent Limitations for Compositional Zero-Shot Learning
  15 May 2025 1 repository listed Syntology ran 23 of 28 samples · 5 unverified · 28 pointer-only (licence)
  To address this issue, we propose a Multi-Stage Cross-modal Interaction (MSCI) model that effectively explores and utilizes intermediate-layer information from CLIP's visual encoder.
• Anatomical Attention Alignment representation for Radiology Report Generation
  12 May 2025 1 repository listed
  Automated Radiology report generation (RRG) aims at producing detailed descriptions of medical images, reducing radiologists' workload and improving access to high-quality diagnostic services.
• HCMA: Hierarchical Cross-model Alignment for Grounded Text-to-Image Generation
  10 May 2025 1 repository listed
  Text-to-image synthesis has progressed to the point where models can generate visually compelling images from natural language prompts.
• Task-Adapter++: Task-specific Adaptation with Order-aware Alignment for Few-shot Action Recognition
  9 May 2025 1 repository listed
  Large-scale pre-trained models have achieved remarkable success in language and image tasks, leading an increasing number of studies to explore the application of pre-trained image models, such as CLIP, in the domain of…
• Probabilistic Embeddings for Frozen Vision-Language Models: Uncertainty Quantification with Gaussian Process Latent Variable Models
  8 May 2025 1 repository listed
  Vision-Language Models (VLMs) learn joint representations by mapping images and text into a shared latent space.
• CAV-MAE Sync: Improving Contrastive Audio-Visual Mask Autoencoders via Fine-Grained Alignment
  2 May 2025 1 repository listed Syntology ran 1 of 2 samples · 1 unverified
  Second, we resolve conflicting optimization goals by separating contrastive and reconstruction objectives through dedicated global tokens.
• MicarVLMoE: A Modern Gated Cross-Aligned Vision-Language Mixture of Experts Model for Medical Image Captioning and Report Generation
  29 Apr 2025 1 repository listed
  Medical image reporting (MIR) aims to generate structured clinical descriptions from radiological images.
• Cross-attention for State-based model RWKV-7
  19 Apr 2025 1 repository listed
  We introduce CrossWKV, a novel cross-attention mechanism for the state-based RWKV-7 model, designed to enhance the expressive power of text-to-image generation.
• 3D CoCa: Contrastive Learners are 3D Captioners
  13 Apr 2025 1 repository listed
  3D captioning, which aims to describe the content of 3D scenes in natural language, remains highly challenging due to the inherent sparsity of point clouds and weak cross-modal alignment in existing methods.
• Gaze-Guided Learning: Avoiding Shortcut Bias in Visual Classification
  8 Apr 2025 1 repository listed
  Motivated by this, we introduce Gaze-CIFAR-10, a human gaze time-series dataset, along with a dual-sequence gaze encoder that models the precise sequential localization of human attention on distinct local attributes.
• Multimodal Fusion and Vision-Language Models: A Survey for Robot Vision
  3 Apr 2025 1 repository listed
  Robot vision has greatly benefited from advancements in multimodal fusion techniques and vision-language models (VLMs).
• COST: Contrastive One-Stage Transformer for Vision-Language Small Object Tracking
  2 Apr 2025 1 repository listed
  In this work, we propose COST, a contrastive one-stage transformer fusion framework for VL tracking, aiming to learn semantically consistent and unified VL representations.
• BiPVL-Seg: Bidirectional Progressive Vision-Language Fusion with Global-Local Alignment for Medical Image Segmentation
  30 Mar 2025 1 repository listed
  Medical image segmentation typically relies solely on visual data, overlooking the rich textual information clinicians use for diagnosis.
• LPOSS: Label Propagation Over Patches and Pixels for Open-vocabulary Semantic Segmentation
  25 Mar 2025 1 repository listed Syntology ran 0 of 5 samples · 5 unverified
  We propose a training-free method for open-vocabulary semantic segmentation using Vision-and-Language Models (VLMs).
• Enhanced OoD Detection through Cross-Modal Alignment of Multi-Modal Representations
  24 Mar 2025 1 repository listed
  To address this, we propose a training objective that enhances cross-modal alignment by regularizing the distances between image and text embeddings of ID data.

Syntology lines on 10 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.