Papers › Gimme Signals: Discriminative signal encoding for multimodal activity recognition
Gimme Signals: Discriminative signal encoding for multimodal activity recognition
Raphael Memmesheimer, Nick Theisen, Dietrich Paulus
We present a simple, yet effective and flexible method for action recognition supporting multiple sensor modalities. Multivariate signal sequences are encoded in an image and are then classified using a recently proposed EfficientNet CNN architecture. Our focus was to find an approach that generalizes well across different sensor modalities without specific adaptions while still achieving good results. We apply our method to 4 action recognition datasets containing skeleton sequences, inertial and motion capturing measurements as well as \wifi fingerprints that range up to 120 action classes. Our method defines the current best CNN-based approach on the NTU RGB+D 120 dataset, lifts the state of the art on the ARIL Wi-Fi dataset by +6.78%, improves the UTD-MHAD inertial baseline by +14.4%, the UTD-MHAD skeleton baseline by 1.13% and achieves 96.11% on the Simitate motion capturing data (80/20 split). We further demonstrate experiments on both, modality fusion on a signal level and signal reduction to prevent the representation from overloading.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Action Recognition | NTU RGB+D 120 | Gimme Signals (AIS) | Accuracy (Cross-Setup) | 70.8 | #18 of 21 | Archive leaderboard | report |
| Action Recognition | NTU RGB+D 120 | Gimme Signals (AIS) | Accuracy (Cross-Subject) | 71.59 | #18 of 21 | Archive leaderboard | report |
| Multimodal Activity Recognition | UTD-MHAD | Gimme Signals (Skeleton, AIS) | Accuracy (CS) | 93.33 | #2 of 5 | Archive leaderboard | report |
| Skeleton Based Action Recognition | NTU RGB+D 120 | Gimme Signals (Skeleton, AIS) | Accuracy (Cross-Setup) | 71.6% | #66 of 83 | Archive leaderboard | report |
| Skeleton Based Action Recognition | NTU RGB+D 120 | Gimme Signals (Skeleton, AIS) | Accuracy (Cross-Subject) | 70.8% | #66 of 83 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections