Multimodal Large Language Model

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

1 dataset whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 160 papers with code (347 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models
  23 Jun 2023 4 repositories listed
  Multimodal Large Language Model (MLLM) relies on the powerful LLM to perform multimodal tasks, showing amazing emergent abilities in recent studies, such as writing poems based on an image.
• ShapeLLM: Universal 3D Object Understanding for Embodied Interaction
  27 Feb 2024 3 repositories listed Syntology ran 9 of 17 samples · 8 unverified
  This paper presents ShapeLLM, the first 3D Multimodal Large Language Model (LLM) designed for embodied interaction, exploring a universal 3D object understanding with 3D point clouds and languages.
• VARGPT: Unified Understanding and Generation in a Visual Autoregressive Multimodal Large Language Model
  21 Jan 2025 2 repositories listed
  We present VARGPT, a novel multimodal large language model (MLLM) that unifies visual understanding and generation within a single autoregressive framework.
• Remote Sensing Temporal Vision-Language Models: A Comprehensive Survey
  3 Dec 2024 2 repositories listed
  This survey fills a critical gap in the literature by providing an integrated overview of RSTVLM, offering a foundation for further advancements in remote sensing temporal image understanding.
• Learn from Downstream and Be Yourself in Multimodal Large Language Model Fine-Tuning
  17 Nov 2024 2 repositories listed
  To balance the trade-off between generalization and specialization, we propose measuring the parameter importance for both pre-trained and fine-tuning distributions, based on frozen pre-trained weight magnitude and…
• Baichuan-Omni Technical Report
  11 Oct 2024 2 repositories listed
  The salient multimodal capabilities and interactive experience of GPT-4o highlight its critical role in practical applications, yet it lacks a high-performing open-source counterpart.
• Ovis: Structural Embedding Alignment for Multimodal Large Language Model
  31 May 2024 2 repositories listed
  However, the misalignment between two embedding strategies in MLLMs -- the structural textual embeddings based on an embedding look-up table and the continuous embeddings generated directly by the vision encoder --…
• MiniGPT4-Video: Advancing Multimodal LLMs for Video Understanding with Interleaved Visual-Textual Tokens
  4 Apr 2024 2 repositories listed Syntology ran 2 of 2 samples · 0 unverified
  This paper introduces MiniGPT4-Video, a multimodal Large Language Model (LLM) designed specifically for video understanding.
• Jailbreaking Attack against Multimodal Large Language Model
  4 Feb 2024 2 repositories listed Syntology ran 4 of 4 samples · 0 unverified · 3 pointer-only (licence)
  This paper focuses on jailbreaking attacks against multi-modal large language models (MLLMs), seeking to elicit MLLMs to generate objectionable responses to harmful user queries.
• MLLM-Tool: A Multimodal Large Language Model For Tool Agent Learning
  19 Jan 2024 2 repositories listed Syntology ran 8 of 11 samples · 3 unverified
  Recently, the astonishing performance of large language models (LLMs) in natural language comprehension and generation tasks triggered lots of exploration of using them as central controllers to build agent systems.
• TinyGPT-V: Efficient Multimodal Large Language Model via Small Backbones
  28 Dec 2023 2 repositories listed Syntology ran 7 of 7 samples · 0 unverified
  In recent years, multimodal large language models (MLLMs) such as GPT-4V have demonstrated remarkable advancements, excelling in a variety of vision-language tasks.
• TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding
  4 Dec 2023 2 repositories listed Syntology ran 7 of 11 samples · 4 unverified
  This work proposes TimeChat, a time-sensitive multimodal large language model specifically designed for long video understanding.
• Ferret: Refer and Ground Anything Anywhere at Any Granularity
  11 Oct 2023 2 repositories listed Syntology ran 7 of 8 samples · 1 unverified · 8 pointer-only (licence)
  We introduce Ferret, a new Multimodal Large Language Model (MLLM) capable of understanding spatial referring of any shape or granularity within an image and accurately grounding open-vocabulary descriptions.
• Kosmos-2: Grounding Multimodal Large Language Models to the World
  26 Jun 2023 2 repositories listed
  We introduce Kosmos-2, a Multimodal Large Language Model (MLLM), enabling new capabilities of perceiving object descriptions (e.
• MFGDiffusion: Mask-Guided Smoke Synthesis for Enhanced Forest Fire Detection
  15 Jul 2025 1 repository listed
  Smoke is the first visible indicator of a wildfire.With the advancement of deep learning, image-based smoke detection has become a crucial method for detecting and preventing forest fires.
• ThinkSound: Chain-of-Thought Reasoning in Multimodal Large Language Models for Audio Generation and Editing
  26 Jun 2025 1 repository listed
  While end-to-end video-to-audio generation has greatly improved, producing high-fidelity audio that authentically captures the nuances of visual content remains challenging.
• OracleFusion: Assisting the Decipherment of Oracle Bone Script with Structurally Constrained Semantic Typography
  26 Jun 2025 1 repository listed
  As one of the earliest ancient languages, Oracle Bone Script (OBS) encapsulates the cultural records and intellectual expressions of ancient civilizations.
• MedTVT-R1: A Multimodal LLM Empowering Medical Reasoning and Diagnosis
  23 Jun 2025 1 repository listed
  Accurate and interpretable multi-disease diagnosis remains a critical challenge in medical research, particularly when leveraging heterogeneous multimodal medical data.
• ShareGPT-4o-Image: Aligning Multimodal Models with GPT-4o-Level Image Generation
  22 Jun 2025 1 repository listed Syntology ran 3 of 3 samples · 0 unverified · 3 pointer-only (licence)
  Recent advances in multimodal generative models have unlocked photorealistic, instruction-aligned image generation, yet leading systems like GPT-4o-Image remain proprietary and inaccessible.
• The Condition Number as a Scale-Invariant Proxy for Information Encoding in Neural Units
  19 Jun 2025 1 repository listed
  This paper explores the relationship between the condition number of a neural network's weight tensor and the extent of information encoded by the associated processing unit, viewed through the lens of information…
• VIS-Shepherd: Constructing Critic for LLM-based Data Visualization Generation
  16 Jun 2025 1 repository listed Syntology ran 0 of 6 samples · 6 unverified · 6 pointer-only (licence)
  Data visualization generation using Large Language Models (LLMs) has shown promising results but often produces suboptimal visualizations that require human intervention for improvement.
• Period-LLM: Extending the Periodic Capability of Multimodal Large Language Model
  30 May 2025 1 repository listed
  Periodic or quasi-periodic phenomena reveal intrinsic characteristics in various natural processes, such as weather patterns, movement behaviors, traffic flows, and biological signals.
• un²CLIP: Improving CLIP's Visual Detail Capturing Ability via Inverting unCLIP
  30 May 2025 1 repository listed
  Compared to discriminative models like CLIP, generative models are better at capturing image details because they are trained to learn the data distribution of images.
• Cross-modal RAG: Sub-dimensional Retrieval-Augmented Text-to-Image Generation
  28 May 2025 1 repository listed
  Text-to-image generation increasingly demands access to domain-specific, fine-grained, and rapidly evolving knowledge that pretrained models cannot fully capture.
• GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolution
  27 May 2025 1 repository listed
  To address data scarcity, we introduce SuperRS-VQA (avg.
• Multimodal LLM-Guided Semantic Correction in Text-to-Image Diffusion
  26 May 2025 1 repository listed
  Diffusion models have become the mainstream architecture for text-to-image generation, achieving remarkable progress in visual quality and prompt controllability.
• Diagnosing and Mitigating Modality Interference in Multimodal Large Language Models
  26 May 2025 1 repository listed Syntology ran 3 of 3 samples · 0 unverified · 3 pointer-only (licence)
  Multimodal Large Language Models (MLLMs) have demonstrated impressive capabilities across tasks, yet they often exhibit difficulty in distinguishing task-relevant from irrelevant signals, particularly in tasks like…
• Unifying Multimodal Large Language Model Capabilities and Modalities via Model Merging
  26 May 2025 1 repository listed Syntology ran 5 of 5 samples · 0 unverified · 5 pointer-only (licence)
  While foundation models update slowly due to resource-intensive training requirements, domain-specific models evolve between updates.
• ChemMLLM: Chemical Multimodal Large Language Model
  22 May 2025 1 repository listed
  To fill this gap, in this paper, we propose ChemMLLM, a unified chemical multimodal large language model for molecule understanding and generation.
• Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding
  22 May 2025 1 repository listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  We observe that training with a purely discrete diffusion approach leads to significant training instability, suboptimal performance, and severe length bias issues.

Syntology lines on 12 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.