Papers › Prompt Guided Transformer for Multi-Task Dense Prediction

Prompt Guided Transformer for Multi-Task Dense Prediction

28 Jul 2023arXiv:2307.15362archive 2025-07-28

Yuxiang Lu, Shalayiding Sirejiding, Yue Ding, Chunlin Wang, Hongtao Lu

Task-conditional architecture offers advantage in parameter efficiency but falls short in performance compared to state-of-the-art multi-decoder methods. How to trade off performance and model parameters is an important and difficult problem. In this paper, we introduce a simple and lightweight task-conditional model called Prompt Guided Transformer (PGT) to optimize this challenge. Our approach designs a Prompt-conditioned Transformer block, which incorporates task-specific prompts in the self-attention mechanism to achieve global dependency modeling and parameter-efficient feature adaptation across multiple tasks. This block is integrated into both the shared encoder and decoder, enhancing the capture of intra- and inter-task features. Moreover, we design a lightweight decoder to further reduce parameter usage, which accounts for only 2.7% of the total model parameters. Extensive experiments on two multi-task dense prediction benchmarks, PASCAL-Context and NYUD-v2, demonstrate that our approach achieves state-of-the-art results among task-conditional methods while using fewer parameters, and maintains a significant balance between performance and parameter size.

Paper PDF Code

Code

innovator-zero/MTDP_Lib officialpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Boundary Detection Decoder Monocular Depth Estimation Multi-Task Learning Prediction Semantic Segmentation Surface Normal Estimation

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Boundary Detection	NYU-Depth V2	PGT (Swin-S)	odsF	78.04	#2 of 3	Archive leaderboard	report
Boundary Detection	NYU-Depth V2	PGT (Swin-T)	odsF	77.05	#3 of 3	Archive leaderboard	report
Monocular Depth Estimation	NYU-Depth V2	PGT (Swin-S)	RMSE	0.5468	#77 of 85	Archive leaderboard	report
Monocular Depth Estimation	NYU-Depth V2	PGT (Swin-T)	RMSE	0.59	#82 of 85	Archive leaderboard	report
Semantic Segmentation	NYU Depth v2	PGT (Swin-S)	Mean IoU	46.43	#90 of 121	Archive leaderboard	report
Semantic Segmentation	NYU Depth v2	PGT (Swin-T)	Mean IoU	41.61	#106 of 121	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Absolute Position Encodings Adam Attention BPE Dense Connections Dropout Label Smoothing Layer Normalization Linear Layer Multi-Head Attention Position-Wise Feed-Forward Layer Residual Connection Softmax Transformer

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections