Papers › Continual Spatio-Temporal Graph Convolutional Networks

Continual Spatio-Temporal Graph Convolutional Networks

21 Mar 2022arXiv:2203.11009archive 2025-07-28

Lukas Hedegaard, Negar Heidari, Alexandros Iosifidis

Graph-based reasoning over skeleton data has emerged as a promising approach for human action recognition. However, the application of prior graph-based methods, which predominantly employ whole temporal sequences as their input, to the setting of online inference entails considerable computational redundancy. In this paper, we tackle this issue by reformulating the Spatio-Temporal Graph Convolutional Neural Network as a Continual Inference Network, which can perform step-by-step predictions in time without repeat frame processing. To evaluate our method, we create a continual version of ST-GCN, CoST-GCN, alongside two derived methods with different self-attention mechanisms, CoAGCN and CoS-TR. We investigate weight transfer strategies and architectural modifications for inference acceleration, and perform experiments on the NTU RGB+D 60, NTU RGB+D 120, and Kinetics Skeleton 400 datasets. Retaining similar predictive accuracy, we observe up to 109x reduction in time complexity, on-hardware accelerations of 26x, and reductions in maximum allocated memory of 52% during online inference.

PaperPDFCode

Code

lukashedegaard/continual-skeletons officialmentioned in papermentioned on GitHub report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

Action RecognitionSkeleton Based Action RecognitionTemporal Action LocalizationTemporal Sequences

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Skeleton Based Action Recognition Kinetics-Skeleton dataset AGCN (2-stream) Accuracy 36.9 #17 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset AGCN (2-stream) GFLOPS per prediction 26.91 #17 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset AGCN (1-stream) Accuracy 35 #20 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset AGCN (1-stream) GFLOPS per prediction 13.45 #20 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset S-TR (2-stream) Accuracy 34.7 #23 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset S-TR (2-stream) GFLOPS per prediction 23.24 #23 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset ST-GCN (2-stream) Accuracy 34.4 #24 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset ST-GCN (2-stream) GFLOPS per prediction 24.09 #24 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset ST-GCN (1-stream) Accuracy 33.4 #28 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset ST-GCN (1-stream) GFLOPS per prediction 12.04 #28 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoST-GCN (2-stream) Accuracy 33.1 #29 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoST-GCN (2-stream) GFLOPS per prediction 0.32 #29 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoAGCN (1-stream) Accuracy 33 #30 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoAGCN (1-stream) GFLOPS per prediction 0.18 #30 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoS-TR (2-stream) Accuracy 32.7 #31 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoS-TR (2-stream) GFLOPS per prediction 0.31 #31 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoST-GCN* (2-stream) Accuracy 32.2 #32 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoST-GCN* (2-stream) GFLOPS per prediction 0.22 #32 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset S-TR (1-stream) Accuracy 32 #33 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset S-TR (1-stream) GFLOPS per prediction 11.62 #33 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoST-GCN (1-stream) Accuracy 31.8 #34 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoST-GCN (1-stream) GFLOPS per prediction 0.16 #34 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoST-GCN* (1-stream) Accuracy 30.2 #36 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoST-GCN* (1-stream) GFLOPS per prediction 0.11 #36 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoS-TR* (2-stream) Accuracy 29.9 #37 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoS-TR* (2-stream) GFLOPS per prediction 0.22 #37 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoS-TR (1-stream) Accuracy 29.7 #38 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoAGCN* (2-stream) Accuracy 27.5 #39 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoAGCN* (2-stream) GFLOPS per prediction 0.25 #39 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoS-TR* (1-stream) Accuracy 27.4 #40 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoS-TR* (1-stream) GFLOPS per prediction 0.11 #40 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoAGCN* (1-stream) Accuracy 23.3 #41 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoAGCN* (1-stream) GFLOPS per prediction 0.12 #41 of 42 Archive leaderboard report
Skeleton Based Action Recognition Kinetics-Skeleton dataset CoAGCN (2-stream) GFLOPS per prediction 0.36 #42 of 42 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D CoS-TR* (2-stream) Accuracy (CS) 88.9 #68 of 135 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D CoS-TR* (2-stream) Accuracy (CV) 94.8 #68 of 135 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D CoS-TR* (2-stream) GFLOPs per pred 0.3 #68 of 135 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D CoST-GCN* (2-stream) Accuracy (CS) 88.3 #72 of 135 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D CoST-GCN* (2-stream) Accuracy (CV) 95 #72 of 135 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D CoST-GCN* (2-stream) GFLOPs per pred 0.32 #72 of 135 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D CoST-GCN* Accuracy (CS) 86.3 #85 of 135 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D CoST-GCN* Accuracy (CV) 93.8 #85 of 135 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D CoST-GCN* GFLOPs per pred 0.16 #85 of 135 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D CoS-TR* Accuracy (CS) 86.3 #86 of 135 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D CoS-TR* Accuracy (CV) 92.4 #86 of 135 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D CoS-TR* GFLOPs per pred 0.15 #86 of 135 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D ST-GCN Accuracy (CS) 86 #88 of 135 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D ST-GCN Accuracy (CV) 93.4 #88 of 135 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D ST-GCN GFLOPs per pred 16.73 #88 of 135 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D CoAGCN* (2-stream) Accuracy (CS) 86.0 #89 of 135 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D CoAGCN* (2-stream) Accuracy (CV) 93.1 #89 of 135 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D CoAGCN* (2-stream) GFLOPs per pred 0.44 #89 of 135 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D CoAGCN* Accuracy (CS) 84.1 #100 of 135 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D CoAGCN* Accuracy (CV) 92.6 #100 of 135 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 S-TR (2-stream) Accuracy (Cross-Setup) 86.2 #46 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 S-TR (2-stream) Accuracy (Cross-Subject) 84.8 #46 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 S-TR (2-stream) GFLOPS per prediction 32.4 #46 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 CoS-TR* (2-stream) Accuracy (Cross-Setup) 86.1 #47 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 CoS-TR* (2-stream) Accuracy (Cross-Subject) 84.8 #47 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 CoS-TR* (2-stream) GFLOPS per prediction 0.3 #47 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 CoST-GCN* (2-stream) Accuracy (Cross-Setup) 85.5 #50 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 CoST-GCN* (2-stream) Accuracy (Cross-Subject) 84.0 #50 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 CoST-GCN* (2-stream) GFLOPS per prediction 0.32 #50 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 AGCN (2-stream) Accuracy (Cross-Setup) 85.4 #51 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 AGCN (2-stream) Accuracy (Cross-Subject) 84 #51 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 AGCN (2-stream) GFLOPS per prediction 37.38 #51 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 ST-GCN (2-stream) Accuracy (Cross-Setup) 85.1 #52 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 ST-GCN (2-stream) Accuracy (Cross-Subject) 83.7 #52 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 ST-GCN (2-stream) GFLOPS per prediction 33.46 #52 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 CoAGCN* (2-stream) Accuracy (Cross-Setup) 82 #58 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 CoAGCN* (2-stream) Accuracy (Cross-Subject) 80.4 #58 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 CoAGCN* (2-stream) GFLOPS per prediction 0.44 #58 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 S-TR (1-stream) Accuracy (Cross-Setup) 81.8 #59 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 S-TR (1-stream) Accuracy (Cross-Subject) 80.2 #59 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 S-TR (1-stream) GFLOPS per prediction 16.2 #59 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 CoS-TR* (1-stream) Accuracy (Cross-Setup) 81.7 #60 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 CoS-TR* (1-stream) Accuracy (Cross-Subject) 79.7 #60 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 CoS-TR* (1-stream) GFLOPS per prediction 0.15 #60 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 AGCN (1-stream) Accuracy (Cross-Setup) 80.7 #61 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 AGCN (1-stream) Accuracy (Cross-Subject) 79.7 #61 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 AGCN (1-stream) GFLOPS per prediction 18.69 #61 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 CoST-GCN* (1-stream) Accuracy (Cross-Setup) 81.6 #62 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 CoST-GCN* (1-stream) Accuracy (Cross-Subject) 79.4 #62 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 CoST-GCN* (1-stream) GFLOPS per prediction 0.16 #62 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 ST-GCN (1-stream) Accuracy (Cross-Subject) 79 #63 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 ST-GCN (1-stream) GFLOPS per prediction 16.73 #63 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 CoAGCN* (1-stream) Accuracy (Cross-Setup) 79.1 #65 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 CoAGCN* (1-stream) Accuracy (Cross-Subject) 77.3 #65 of 83 Archive leaderboard report
Skeleton Based Action Recognition NTU RGB+D 120 CoAGCN* (1-stream) GFLOPS per prediction 0.22 #65 of 83 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections