Visual Prompting

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

No dataset record in the archive lists this task.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Parent tasks archive 2025-07-28

Most implemented papers archive 2025-07-28

30 shown of 60 papers with code (127 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Segment Anything
  5 Apr 2023 32 repositories listed Syntology ran 8 of 23 samples · 15 unverified
  We introduce the Segment Anything (SA) project: a new task, model, and dataset for image segmentation.
• ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Prompts
  1 Dec 2023 4 repositories listed
  Furthermore, we present ViP-Bench, a comprehensive benchmark to assess the capability of models in understanding visual prompts across multiple dimensions, enabling future research in this domain.
• Set-of-Mark Prompting Unleashes Extraordinary Visual Grounding in GPT-4V
  17 Oct 2023 4 repositories listed Syntology ran 0 of 1 samples · 1 unverified
  We present Set-of-Mark (SoM), a new visual prompting method, to unleash the visual grounding abilities of large multimodal models (LMMs), such as GPT-4V.
• Visual In-Context Prompting
  22 Nov 2023 3 repositories listed Syntology ran 3 of 6 samples · 3 unverified · 6 pointer-only (licence)
  In-context prompting in large language models (LLMs) has become a prevalent approach to improve zero-shot capabilities, but this idea is less explored in the vision domain.
• Uncovering the Hidden Cost of Model Compression
  29 Aug 2023 2 repositories listed Syntology ran 2 of 2 samples · 0 unverified · 2 pointer-only (licence)
  This empirical investigation underscores the need for a nuanced understanding beyond mere accuracy in sparse and quantized settings, thereby paving the way for further exploration in Visual Prompting techniques tailored…
• Explicit Visual Prompting for Universal Foreground Segmentations
  29 May 2023 2 repositories listed
  We take inspiration from the widely-used pre-training and then prompt tuning protocols in NLP and propose a new visual prompting model, named Explicit Visual Prompting (EVP).
• Visual Prompting for Adversarial Robustness
  12 Oct 2022 2 repositories listed Syntology ran 1 of 1 samples · 0 unverified
  In this work, we leverage visual prompting (VP) to improve adversarial robustness of a fixed, pre-trained model at testing time.
• Stepwise Decomposition and Dual-stream Focus: A Novel Approach for Training-free Camouflaged Object Segmentation
  7 Jun 2025 1 repository listed
  MSD-CoT progressively disentangles image captions to eliminate semantic ambiguity, while RDVP injects spatial constraints into visual prompting and independently samples visual prompts for foreground and background…
• Vision Graph Prompting via Semantic Low-Rank Decomposition
  7 May 2025 1 repository listed Syntology ran 1 of 2 samples · 1 unverified · 2 pointer-only (licence)
  Vision GNN (ViG) demonstrates superior performance by representing images as graph structures, providing a more natural way to capture irregular semantic patterns beyond traditional grid or sequence-based…
• Token Coordinated Prompt Attention is Needed for Visual Prompting
  5 May 2025 1 repository listed
  Extensive experiments across various benchmarks demonstrate that TCPA significantly enhances the diversity and discriminative power of the extracted features.
• Towards Online Multi-Modal Social Interaction Understanding
  25 Mar 2025 1 repository listed
  To address the challenges of missing the useful future context, we develop a novel framework, named Online-MMSI-VLM, that leverages two complementary strategies: multi-party conversation forecasting and social-aware…
• Chameleon: Fast-slow Neuro-symbolic Lane Topology Extraction
  10 Mar 2025 1 repository listed
  Lane topology extraction involves detecting lanes and traffic elements and determining their relationships, a key perception task for mapless autonomous driving.
• Towards Ambiguity-Free Spatial Foundation Model: Rethinking and Decoupling Depth Ambiguity
  8 Mar 2025 1 repository listed
  Depth ambiguity is a fundamental challenge in spatial scene understanding, especially in transparent scenes where single-depth estimates fail to capture full 3D structure.
• LoR-VP: Low-Rank Visual Prompting for Efficient Vision Model Adaptation
  2 Feb 2025 1 repository listed
  Visual prompting has gained popularity as a method for adapting pre-trained models to specific tasks, particularly in the realm of parameter-efficient tuning.
• IP-Prompter: Training-Free Theme-Specific Image Generation via Dynamic Visual Prompting
  26 Jan 2025 1 repository listed
  The stories and characters that captivate us as we grow up shape unique fantasy worlds, with images serving as the primary medium for visually experiencing these realms.
• GPT4Scene: Understand 3D Scenes from Videos with Vision-Language Models
  2 Jan 2025 1 repository listed Syntology ran 1 of 8 samples · 7 unverified
  To address this, we introduce GPT4Scene, a novel visual prompting paradigm in VLM training and inference that helps build the global-local relationship, significantly improving the 3D spatial understanding of indoor…
• Selective Visual Prompting in Vision Mamba
  12 Dec 2024 1 repository listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  To prevent the loss of discriminative information during state space propagation, SVP employs lightweight selective prompters for token-wise prompt generation, ensuring adaptive activation of the update and forget gates…
• Inst-IT: Boosting Multimodal Instance Understanding via Explicit Visual Prompt Instruction Tuning
  4 Dec 2024 1 repository listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  Building upon this pipeline, we proposed Inst-IT, a solution to enhance LMMs in Instance understanding via explicit visual prompt Instruction Tuning.
• Improved GUI Grounding via Iterative Narrowing
  18 Nov 2024 1 repository listed
  Graphical User Interface (GUI) grounding plays a crucial role in enhancing the capabilities of Vision-Language Model (VLM) agents.
• Benchmarking Human and Automated Prompting in the Segment Anything Model
  29 Oct 2024 1 repository listed
  Additionally, we demonstrate that performance when using automated methods can be improved by up to 68% via a finetuning approach.
• Improving Visual Object Tracking through Visual Prompting
  27 Sep 2024 1 repository listed
  PiVOT proposes a prompt generation network with the pre-trained foundation model CLIP to automatically generate and refine visual prompts, enabling the transfer of foundation model knowledge for tracking.
• Attention Prompting on Image for Large Vision-Language Models
  25 Sep 2024 1 repository listed Syntology ran 8 of 12 samples · 4 unverified · 2 pointer-only (licence)
  To fill this gap, in this work, we propose a new prompting technique named Attention Prompting on Image, which just simply overlays a text-query-guided attention heatmap on the original input image and effectively…
• When Does Visual Prompting Outperform Linear Probing for Vision-Language Models? A Likelihood Perspective
  3 Sep 2024 1 repository listed
  Adapting pre-trained models to new tasks can exhibit varying effectiveness across datasets.
• Open-Vocabulary Action Localization with Iterative Visual Prompting
  30 Aug 2024 1 repository listed Syntology ran 1 of 1 samples · 0 unverified
  Video action localization aims to find the timings of specific actions from a long video.
• Targeted Visual Prompting for Medical Visual Question Answering
  6 Aug 2024 1 repository listed
  With growing interest in recent years, medical visual question answering (Med-VQA) has rapidly evolved, with multimodal large language models (MLLMs) emerging as an alternative to classical model architectures.
• EarthMarker: A Visual Prompting Multi-modal Large Language Model for Remote Sensing
  18 Jul 2024 1 repository listed
  Specifically, a shared visual encoding method is developed to establish the spatial pattern interpretation relationships between the multi-scale representations of input images and various visual prompts.
• By My Eyes: Grounding Multimodal Large Language Models with Sensor Data via Visual Prompting
  15 Jul 2024 1 repository listed
  We design a visual prompt that directs MLLMs to utilize visualized sensor data alongside the target sensory task descriptions.
• UICrit: Enhancing Automated Design Evaluation with a UICritique Dataset
  11 Jul 2024 1 repository listed
  We hypothesize that automatic evaluation can be improved by collecting a targeted UI feedback dataset and then using this dataset to enhance the performance of general-purpose LLMs.
• Dynamic Domains, Dynamic Solutions: DPCore for Continual Test-Time Adaptation
  15 Jun 2024 1 repository listed Syntology ran 8 of 11 samples · 3 unverified
  Continual Test-Time Adaptation (CTTA) seeks to adapt a source pre-trained model to continually changing, unlabeled target domains.
• OT-VP: Optimal Transport-guided Visual Prompting for Test-Time Adaptation
  12 Jun 2024 1 repository listed Syntology ran 12 of 16 samples · 4 unverified
  Vision Transformers (ViTs) have demonstrated remarkable capabilities in learning representations, but their performance is compromised when applied to unseen domains.

Syntology lines on 13 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.