Papers › A Hybrid CNN-BiLSTM Voice Activity Detector

A Hybrid CNN-BiLSTM Voice Activity Detector

5 Mar 2021arXiv:2103.03529archive 2025-07-28

Nicholas Wilkinson, Thomas Niesler

This paper presents a new hybrid architecture for voice activity detection (VAD) incorporating both convolutional neural network (CNN) and bidirectional long short-term memory (BiLSTM) layers trained in an end-to-end manner. In addition, we focus specifically on optimising the computational efficiency of our architecture in order to deliver robust performance in difficult in-the-wild noise conditions in a severely under-resourced setting. Nested k-fold cross-validation was used to explore the hyperparameter space, and the trade-off between optimal parameters and model size is discussed. The performance effect of a BiLSTM layer compared to a unidirectional LSTM layer was also considered. We compare our systems with three established baselines on the AVA-Speech dataset. We find that significantly smaller models with near optimal parameters perform on par with larger models trained with optimal parameters. BiLSTM layers were shown to improve accuracy over unidirectional layers by ≈2% absolute on average. With an area under the curve (AUC) of 0.951, our system outperforms all baselines, including a much larger ResNet system, particularly in difficult noise conditions.

Paper PDF Code

Code

NickWilkinson37/voxseg officialmentioned on GitHubtf report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Action Detection Activity Detection Computational Efficiency

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Activity Detection	AVA-Speech	CNN-BiLSTM_best	ROC-AUC	95.14	#1 of 4	Archive leaderboard	report
Activity Detection	AVA-Speech	CNN-BiLSTM_small	ROC-AUC	95.13	#2 of 4	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

1x1 Convolution Average Pooling Batch Normalization BiLSTM Bottleneck Residual Block Convolution Global Average Pooling Kaiming Initialization LSTM Max Pooling ReLU Residual Block Residual Connection Sigmoid Activation Tanh Activation

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections