Papers › ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition...

ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context

7 May 2020arXiv:2005.03191archive 2025-07-28

Wei Han, Zhengdong Zhang, Yu Zhang, Jiahui Yu, Chung-Cheng Chiu, James Qin, Anmol Gulati, Ruoming Pang, Yonghui Wu

Convolutional neural networks (CNN) have shown promising results for end-to-end speech recognition, albeit still behind other state-of-the-art methods in performance. In this paper, we study how to bridge this gap and go beyond with a novel CNN-RNN-transducer architecture, which we call ContextNet. ContextNet features a fully convolutional encoder that incorporates global context information into convolution layers by adding squeeze-and-excitation modules. In addition, we propose a simple scaling method that scales the widths of ContextNet that achieves good trade-off between computation and accuracy. We demonstrate that on the widely used LibriSpeech benchmark, ContextNet achieves a word error rate (WER) of 2.1%/4.6% without external language model (LM), 1.9%/4.1% with LM and 2.9%/7.0% with only 10M parameters on the clean/noisy LibriSpeech test sets. This compares to the previous best published system of 2.0%/4.6% with LM and 3.9%/11.3% with 20M parameters. The superiority of the proposed ContextNet model is also verified on a much larger internal dataset.

PaperPDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

Cross-Caps/STFADE mentioned on GitHubtfGPL-3.0Syntology: not harvested report
TensorSpeech/TensorFlowASR mentioned on GitHubtfApache-2.0Syntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Conformer: Convolution-augmented Transformer for Speech Recognition (arXiv:2005.08100; that paper's own run record: 6 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 3 violated, 2 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified); nothing checks that they implement this paper's method. report
hasangchun/ContextNet mentioned on GitHubpytorchApache-2.0Syntology: not harvested report
msalhab96/SpeeQ mentioned on GitHubpytorchSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to Squeezeformer: An Efficient Transformer for Automatic Speech Recognition (arXiv:2206.00888; that paper's own run record: 31 ran (of which 12 constructed an object rather than computing a result; 28 with no instrument failure: 3 honoured, 0 violated, 25 with no contract checked; 3 where Syntology's instrument failed) · 18 unverified); nothing checks that they implement this paper's method. report
openspeech-team/openspeech mentioned on GitHubpytorchMITSyntology: not harvested report
upskyy/ContextNet mentioned on GitHubpytorchApache-2.0Syntology: not harvested report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Language ModelingLanguage ModellingSpeech Recognitionspeech-recognition

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Speech Recognition LibriSpeech test-clean ContextNet(L) Word Error Rate (WER) 1.9 #17 of 64 Archive leaderboard report
Speech Recognition LibriSpeech test-clean ContextNet(M) Word Error Rate (WER) 2 #20 of 64 Archive leaderboard report
Speech Recognition LibriSpeech test-clean ContextNet(S) Word Error Rate (WER) 2.3 #36 of 64 Archive leaderboard report
Speech Recognition LibriSpeech test-other ContextNet(L) Word Error Rate (WER) 4.1 #18 of 53 Archive leaderboard report
Speech Recognition LibriSpeech test-other ContextNet(M) Word Error Rate (WER) 4.5 #27 of 53 Archive leaderboard report
Speech Recognition LibriSpeech test-other ContextNet(S) Word Error Rate (WER) 5.5 #33 of 53 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Convolution

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections