Papers › GASP: Gated Attention For Saliency Prediction

GASP: Gated Attention For Saliency Prediction

9 Jun 2022International Joint Conference on Artificial Intelligence 2021 8arXiv:2206.04590archive 2025-07-28

Fares Abawi, Tom Weber, Stefan Wermter

Saliency prediction refers to the computational task of modeling overt attention. Social cues greatly influence our attention, consequently altering our eye movements and behavior. To emphasize the efficacy of such features, we present a neural model for integrating social cues and weighting their influences. Our model consists of two stages. During the first stage, we detect two social cues by following gaze, estimating gaze direction, and recognizing affect. These features are then transformed into spatiotemporal maps through image processing operations. The transformed representations are propagated to the second stage (GASP) where we explore various techniques of late fusion for integrating social cues and introduce two sub-networks for directing attention to relevant stimuli. Our experiments indicate that fusion approaches achieve better results for static integration methods, whereas non-fusion approaches for which the influence of each modality is unknown, result in better outcomes when coupled with recurrent models for dynamic saliency prediction. We show that gaze direction and affective representations contribute a prediction to ground-truth correspondence improvement of at least 5% compared to dynamic saliency models without social cues. Furthermore, affective representations improve GASP, supporting the necessity of considering affect-biased attention in predicting saliency.

Paper PDF Conference PDF Code

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

Code

knowledgetechnologyuhh/gasp officialpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Prediction Saliency Prediction Video Saliency Detection Video Saliency Prediction

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Video Saliency Detection	MSU Video Saliency Prediction	GASP	AUC-J	0.810	#10 of 14	Archive leaderboard	report
Video Saliency Detection	MSU Video Saliency Prediction	GASP	CC	0.613	#10 of 14	Archive leaderboard	report
Video Saliency Detection	MSU Video Saliency Prediction	GASP	FPS	3.77	#10 of 14	Archive leaderboard	report
Video Saliency Detection	MSU Video Saliency Prediction	GASP	KLDiv	0.687	#10 of 14	Archive leaderboard	report
Video Saliency Detection	MSU Video Saliency Prediction	GASP	NSS	1.57	#10 of 14	Archive leaderboard	report
Video Saliency Detection	MSU Video Saliency Prediction	GASP	SIM	0.557	#10 of 14	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

1x1 Convolution Average Pooling Convolution Dense Connections Gated Convolution Gated Linear Unit LSTM ReLU Sigmoid Activation Squeeze-and-Excitation Block Tanh Activation

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections