Papers › Multi-scale Bottleneck Transformer for Weakly Supervised Multimodal Violence Detection

Multi-scale Bottleneck Transformer for Weakly Supervised Multimodal Violence Detection

8 May 2024arXiv:2405.05130archive 2025-07-28

Shengyang Sun, Xiaojin Gong

Weakly supervised multimodal violence detection aims to learn a violence detection model by leveraging multiple modalities such as RGB, optical flow, and audio, while only video-level annotations are available. In the pursuit of effective multimodal violence detection (MVD), information redundancy, modality imbalance, and modality asynchrony are identified as three key challenges. In this work, we propose a new weakly supervised MVD method that explicitly addresses these challenges. Specifically, we introduce a multi-scale bottleneck transformer (MSBT) based fusion module that employs a reduced number of bottleneck tokens to gradually condense information and fuse each pair of modalities and utilizes a bottleneck token-based weighting scheme to highlight more important fused features. Furthermore, we propose a temporal consistency contrast loss to semantically align pairwise fused features. Experiments on the largest-scale XD-Violence dataset demonstrate that the proposed method achieves state-of-the-art performance. Code is available at https://github.com/shengyangsun/MSBT.

Paper PDF Code

Code

shengyangsun/MSBT officialmentioned in papermentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Anomaly Detection In Surveillance Videos Optical Flow Estimation

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Anomaly Detection In Surveillance Videos	XD-Violence	MSBT	AP	84.32	#8 of 17	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

1x1 Convolution ALIGN Attention Bottleneck Transformer Bottleneck Transformer Block Convolution Linear Layer Max Pooling Multi-Head Attention Pointwise Convolution Residual Connection Softmax

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections