Papers › A New Deep Learning Architecture withInductive Bias Balance for Transformer Oil...

A New Deep Learning Architecture withInductive Bias Balance for Transformer Oil Temperature Forecasting

28 May 2023Journal of Big Data 2023 5archive 2025-07-28

Manuel Jesús Jiménez-Navarro, María Martínez-Ballesteros, Francisco Martínez-Álvarez & Gualberto Asencio-Cortés

Ensuring optimal performance of power transformers is a laborious task, where the insulation system is essential to decrease their deterioration. The insulation system uses the insulate oil required to control temperature. High temperatures may reduce the lifetime of transformers, leading to expensive maintenance. Deep learning architectures have been shown to obtain remarkable results in a wide range of fields. However, this improvement usually comes with an increase in computing resources, which increases the carbon footprint and hinders the optimization of the architectures. In this work, we develop a new deep learning architecture that obtain an efficacy which compete with the best current architectures in transformer oil temperature forecasting while improve the efficacy. Effective forecasting can help prevent high temperatures and monitor the future condition of power transformers, avoiding unnecessary waste. We attempt to balance the inductive bias included in our architecture through the proposed Smooth Residual Block. This mechanism divides the original problem into multiple subproblems, obtaining different representations of the time series, which collaboratively obtain the final forecasting. Our architecture is applied to the Electricity Transformer datasets, which obtain transformer insulate oil temperature measures from two transformers in China. The results achieve a 13% improvement in MSE and a 57% improvement in performance compared to, as far as we know, the best current architectures. Additionally, we analyze the behavior learned by this architecture to obtain an intuitive interpretation of the achieved solution.

Paper PDF Code

Code

manjimnav/SRCNet mentioned in paperpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Inductive Bias Time Series Time Series Forecasting Univariate Time Series Forecasting

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Time Series Forecasting	ETTh1 (24) Univariate	SRCNet	MAE	0.129	#5 of 5	Archive leaderboard	report
Time Series Forecasting	ETTh2 (24) Univariate	SRCNet	MAE	0.193	#5 of 5	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Absolute Position Encodings Adam Attention BPE Batch Normalization Convolution Dense Connections Dropout Label Smoothing Layer Normalization Linear Layer Multi-Head Attention Position-Wise Feed-Forward Layer ReLU Residual Block Residual Connection Softmax Transformer

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections