Papers › T-Graphormer: Using Transformers for Spatiotemporal Forecasting

T-Graphormer: Using Transformers for Spatiotemporal Forecasting

22 Jan 2025arXiv:2501.13274archive 2025-07-28

Hao Yuan Bai, Xue Liu

Spatiotemporal data is ubiquitous, and forecasting it has important applications in many domains. However, its complex cross-component dependencies and non-linear temporal dynamics can be challenging for traditional techniques. Existing methods address this by learning the two dimensions separately. Here, we introduce Temporal Graphormer (T-Graphormer), a Transformer-based approach capable of modelling spatiotemporal correlations simultaneously. By adding temporal encodings in the Graphormer architecture, each node attends to all other tokens within the graph sequence, enabling the model to learn rich spacetime patterns with minimal predefined inductive biases. We show the effectiveness of T-Graphormer on real-world traffic prediction benchmark datasets. Compared to state-of-the-art methods, T-Graphormer reduces root mean squared error (RMSE) and mean absolute percentage error (MAPE) by up to 20% and 10%.

Paper PDF Code

Code

rdh1115/T-Graphormer officialmentioned on GitHubpytorch report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Time Series Traffic Prediction

Results from the paper archive 2025-07-28

Task	Dataset	Model	Metric	Value	Rank at snapshot	Leaderboard	Report
Traffic Prediction	METR-LA	T-Graphormer	12 steps MAE	3.19	#2 of 20	Archive leaderboard	report
Traffic Prediction	METR-LA	T-Graphormer	12 steps MAPE	8.62	#2 of 20	Archive leaderboard	report
Traffic Prediction	METR-LA	T-Graphormer	12 steps RMSE	6.12	#2 of 20	Archive leaderboard	report
Traffic Prediction	METR-LA	T-Graphormer	MAE @ 12 step	3.19	#2 of 20	Archive leaderboard	report
Traffic Prediction	METR-LA	T-Graphormer	MAE @ 3 step	2.63	#2 of 20	Archive leaderboard	report
Traffic Prediction	PEMS-BAY	T-Graphormer	MAE @ 12 step	1.63	#1 of 16	Archive leaderboard	report
Traffic Prediction	PEMS-BAY	T-Graphormer	RMSE	3.20	#1 of 16	Archive leaderboard	report
Traffic Prediction	PEMS-BAY	T-Graphormer	RMSE	3.20	#1 of 16	Archive leaderboard	report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

Absolute Position Encodings Adam Attention BPE Dense Connections Dropout Label Smoothing Layer Normalization Linear Layer Multi-Head Attention Position-Wise Feed-Forward Layer Residual Connection Softmax Transformer

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections