Methods › General › Miscellaneous Components › Highway Layer
Highway Layer
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
A Highway Layer contains an information highway to other layers that helps with information flow. It is characterised by the use of a gating unit to help this information flow.
A plain feedforward neural network typically consists of L layers where the $l$th layer (l ∈{1, 2, …, L}) applies a nonlinear transform H (parameterized by 𝐖_(𝐇,𝐥)) on its input 𝐱ₗ to produce its output 𝐲ₗ. Thus, 𝐱₁ is the input to the network and 𝐲_𝐋 is the network’s output. Omitting the layer index and biases for clarity,
𝐲 = H(𝐱,𝐖_𝐇)
H is usually an affine transform followed by a non-linear activation function, but in general it may take other forms.
For a highway network, we additionally define two nonlinear transforms T(𝐱,𝐖_𝐓) and C(𝐱,𝐖_𝐂) such that:
𝐲 = H(𝐱,𝐖_𝐇)·T(𝐱,𝐖_𝐓) + 𝐱·C(𝐱,𝐖_𝐂)
We refer to T as the transform gate and C as the carry gate, since they express how much of the output is produced by transforming the input and carrying it, respectively. In the original paper, the authors set C = 1 − T, giving:
𝐲 = H(𝐱,𝐖_𝐇)·T(𝐱,𝐖_𝐓) + 𝐱·(1-T(𝐱,𝐖_𝐓))
The authors set:
T(x) = σ(𝐖_𝐓ᵀ𝐱 + 𝐛_𝐓)
Image: Sik-Ho Tsang
Papers archive 2025-07-28
30 shown of 108, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
Coordinated Ramp Metering Control based on Scalable Nonlinear Traffic Dynamics Model Discovery in a Large Network 9 Mar 2025 · 0 repositories · arXiv:2503.06767
-
Robust and Unbounded Length Generalization in Autoregressive Transformer-Based Text-to-Speech 29 Oct 2024 · 1 repository · arXiv:2410.22179
-
Enhancing Kurdish Text-to-Speech with Native Corpus Training: A High-Quality WaveGlow Vocoder Approach 10 Sep 2024 · 0 repositories · arXiv:2409.13734
-
Training Universal Vocoders with Feature Smoothing-Based Augmentation Methods for High-Quality TTS Systems 4 Sep 2024 · 0 repositories · arXiv:2409.02517
-
Expanding the Scope: Inductive Knowledge Graph Reasoning with Multi-Starting Progressive Propagation 15 Jul 2024 · 1 repository · arXiv:2407.10430
-
Highway Networks for Improved Surface Reconstruction: The Role of Residuals and Weight Updates 11 Jul 2024 · 1 repository · arXiv:2407.08134
-
Stable Weight Updating: A Key to Reliable PDE Solutions Using Deep Learning 10 Jul 2024 · 0 repositories · arXiv:2407.07375
-
Modeling of spatially embedded networks via regional spatial graph convolutional networks 20 Jun 2024 · 1 repository
-
Informed along the road: roadway capacity driven graph convolution network for network-wide traffic prediction 18 Jun 2024 · 0 repositories · arXiv:2406.13057
-
Traffic Prediction considering Multiple Levels of Spatial-temporal Information: A Multi-scale Graph Wavelet-based Approach 18 Jun 2024 · 0 repositories · arXiv:2406.13038
-
Leveraging the Interplay Between Syntactic and Acoustic Cues for Optimizing Korean TTS Pause Formation 3 Apr 2024 · 0 repositories · arXiv:2404.02592
-
Selective-Stereo: Adaptive Frequency Information Selection for Stereo Matching 1 Mar 2024 · 1 repository · arXiv:2403.00486Syntology ran 9 of 13 samples · 4 unverified
-
On the Effectiveness of Unlearning in Session-Based Recommendation 22 Dec 2023 · 1 repository · arXiv:2312.14447Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
-
A Data-Driven Integrated Framework for Fast-Charging Facility Planning using Multi-Period Bi-Objective Optimization 21 Nov 2023 · 0 repositories · arXiv:2311.12700
-
Reinforcement Learning with Model Predictive Control for Highway Ramp Metering 15 Nov 2023 · 1 repository · arXiv:2311.08820
-
Distil the informative essence of loop detector data set: Is network-level traffic forecasting hungry for more data? 31 Oct 2023 · 1 repository · arXiv:2310.20366
-
Power-Enhanced Residual Network for Function Approximation and Physics-Informed Inverse Problems 24 Oct 2023 · 1 repository · arXiv:2310.15690
-
An overview of text-to-speech systems and media applications 22 Oct 2023 · 0 repositories · arXiv:2310.14301
-
Energy-Based Models For Speech Synthesis 19 Oct 2023 · 0 repositories · arXiv:2310.12765
-
Facilitating Battery Swapping Services for Freight Trucks with Spatial-Temporal Demand Prediction 1 Oct 2023 · 0 repositories · arXiv:2310.04440
-
The DeepZen Speech Synthesis System for Blizzard Challenge 2023 30 Aug 2023 · 0 repositories · arXiv:2308.15945
-
Machine learning for option pricing: an empirical investigation of network architectures 14 Jul 2023 · 0 repositories · arXiv:2307.07657
-
Multilingual Text-to-Speech Synthesis for Turkic Languages Using Transliteration 25 May 2023 · 1 repository · arXiv:2305.15749
-
Neural Machine Translation Models with Attention-Based Dropout Layer 1 May 2023 · 1 repository
-
A Virtual Simulation-Pilot Agent for Training of Air Traffic Controllers 16 Apr 2023 · 0 repositories · arXiv:2304.07842
-
ArmanTTS single-speaker Persian dataset 7 Apr 2023 · 0 repositories · arXiv:2304.03585
-
SCConv: Spatial and Channel Reconstruction Convolution for Feature Redundancy 1 Jan 2023 · 1 repository
-
Investigation of Japanese PnG BERT language model in text-to-speech synthesis for pitch accent language 16 Dec 2022 · 0 repositories · arXiv:2212.08321
-
Investigating Content-Aware Neural Text-To-Speech MOS Prediction Using Prosodic and Linguistic Features 1 Nov 2022 · 0 repositories · arXiv:2211.00342
-
Cross-lingual Text-To-Speech with Flow-based Voice Conversion for Improved Pronunciation 31 Oct 2022 · 0 repositories · arXiv:2210.17264
Tasks archive 2025-07-28
20 shown of 96 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections