Papers › Recurrent Glimpse-based Decoder for Detection with Transformer

Recurrent Glimpse-based Decoder for Detection with Transformer

9 Dec 2021CVPR 2022 1arXiv:2112.04632archive 2025-07-28

Zhe Chen, Jing Zhang, DaCheng Tao

Although detection with Transformer (DETR) is increasingly popular, its global attention modeling requires an extremely long training period to optimize and achieve promising detection performance. Alternative to existing studies that mainly develop advanced feature or embedding designs to tackle the training issue, we point out that the Region-of-Interest (RoI) based detection refinement can easily help mitigate the difficulty of training for DETR methods. Based on this, we introduce a novel REcurrent Glimpse-based decOder (REGO) in this paper. In particular, the REGO employs a multi-stage recurrent processing structure to help the attention of DETR gradually focus on foreground objects more accurately. In each processing stage, visual features are extracted as glimpse features from RoIs with enlarged bounding box areas of detection results from the previous stage. Then, a glimpse-based decoder is introduced to provide refined detection results based on both the glimpse features and the attention modeling outputs of the previous stage. In practice, REGO can be easily embedded in representative DETR variants while maintaining their fully end-to-end training and inference pipelines. In particular, REGO helps Deformable DETR achieve 44.8 AP on the MSCOCO dataset with only 36 training epochs, compared with the first DETR and the Deformable DETR that require 500 and 50 epochs to achieve comparable performance, respectively. Experiments also show that REGO consistently boosts the performance of different DETR detectors by up to 7% relative gain at the same setting of 50 training epochs. Code is available via https://github.com/zhechen/Deformable-DETR-REGO.

Paper PDF Conference PDF Code

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

zhechen/deformable-detr-rego officialmentioned in papermentioned on GitHubpytorchApache-2.0 report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Decoder Object Detection

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Object Detection	COCO (Common Objects in Context)	REGO-Deformable DETR-X101	GFlops	434	#3 of 3	Archive leaderboard	report
Object Detection	COCO minival	REGO-Deformable DETR-X101	AP50	67.5	#86 of 220	Archive leaderboard	report
Object Detection	COCO minival	REGO-Deformable DETR-X101	AP75	53.1	#86 of 220	Archive leaderboard	report
Object Detection	COCO minival	REGO-Deformable DETR-X101	APL	65	#86 of 220	Archive leaderboard	report
Object Detection	COCO minival	REGO-Deformable DETR-X101	APM	52.6	#86 of 220	Archive leaderboard	report
Object Detection	COCO minival	REGO-Deformable DETR-X101	APS	30	#86 of 220	Archive leaderboard	report
Object Detection	COCO minival	REGO-Deformable DETR-X101	box AP	49.1	#86 of 220	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Absolute Position Encodings Adam Attention BPE Convolution Deformable Attention Module Deformable DETR Dense Connections Detr Dropout Feedforward Network Label Smoothing Layer Normalization Linear Layer Multi-Head Attention Position-Wise Feed-Forward Layer Residual Connection Softmax Transformer

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections