{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/t-graphormer-using-transformers-for","title":"T-Graphormer: Using Transformers for Spatiotemporal Forecasting","arxiv_id":"2501.13274","date":"2025-01-22","proceeding":null,"authors":["Hao Yuan Bai","Xue Liu"],"abstract":"Spatiotemporal data is ubiquitous, and forecasting it has important applications in many domains. However, its complex cross-component dependencies and non-linear temporal dynamics can be challenging for traditional techniques. Existing methods address this by learning the two dimensions separately. Here, we introduce Temporal Graphormer (T-Graphormer), a Transformer-based approach capable of modelling spatiotemporal correlations simultaneously. By adding temporal encodings in the Graphormer architecture, each node attends to all other tokens within the graph sequence, enabling the model to learn rich spacetime patterns with minimal predefined inductive biases. We show the effectiveness of T-Graphormer on real-world traffic prediction benchmark datasets. Compared to state-of-the-art methods, T-Graphormer reduces root mean squared error (RMSE) and mean absolute percentage error (MAPE) by up to 20% and 10%.","url_abs":"https://arxiv.org/abs/2501.13274v3","url_pdf":"https://arxiv.org/pdf/2501.13274v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"t-graphormer-using-transformers-for","repo_url":"https://github.com/rdh1115/T-Graphormer","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"time-series-1","task_name":"Time Series"},{"task_slug":"traffic-prediction","task_name":"Traffic Prediction"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/traffic-prediction-on-metr-la","task":"Traffic Prediction","dataset":"METR-LA","model":"T-Graphormer","rank_in_archive_order":2,"of":20,"metrics":{"12 steps MAE":"3.19","12 steps MAPE":"8.62","12 steps RMSE":"6.12","MAE @ 12 step":"3.19","MAE @ 3 step":"2.63"},"uses_additional_data":false},{"leaderboard":"/sota/traffic-prediction-on-pems-bay","task":"Traffic Prediction","dataset":"PEMS-BAY","model":"T-Graphormer","rank_in_archive_order":1,"of":16,"metrics":{"MAE @ 12 step":"1.63","RMSE":"3.20","RMSE ":"3.20"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}