{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/stfnets-learning-sensing-signals-from-the","title":"STFNets: Learning Sensing Signals from the Time-Frequency Perspective with Short-Time Fourier Neural Networks","arxiv_id":"1902.07849","date":"2019-02-21","proceeding":null,"authors":["Shuochao Yao","Ailing Piao","Wenjun Jiang","Yiran Zhao","Huajie Shao","Shengzhong Liu","Dongxin Liu","Jinyang Li","Tianshi Wang","Shaohan Hu","Lu Su","Jiawei Han","Tarek Abdelzaher"],"abstract":"Recent advances in deep learning motivate the use of deep neural networks in\nInternet-of-Things (IoT) applications. These networks are modelled after signal\nprocessing in the human brain, thereby leading to significant advantages at\nperceptual tasks such as vision and speech recognition. IoT applications,\nhowever, often measure physical phenomena, where the underlying physics (such\nas inertia, wireless signal propagation, or the natural frequency of\noscillation) are fundamentally a function of signal frequencies, offering\nbetter features in the frequency domain. This observation leads to a\nfundamental question: For IoT applications, can one develop a new brand of\nneural network structures that synthesize features inspired not only by the\nbiology of human perception but also by the fundamental nature of physics?\nHence, in this paper, instead of using conventional building blocks (e.g.,\nconvolutional and recurrent layers), we propose a new foundational neural\nnetwork building block, the Short-Time Fourier Neural Network (STFNet). It\nintegrates a widely-used time-frequency analysis method, the Short-Time Fourier\nTransform, into data processing to learn features directly in the frequency\ndomain, where the physics of underlying phenomena leave better foot-prints.\nSTFNets bring additional flexibility to time-frequency analysis by offering\nnovel nonlinear learnable operations that are spectral-compatible. Moreover,\nSTFNets show that transforming signals to a domain that is more connected to\nthe underlying physics greatly simplifies the learning process. We demonstrate\nthe effectiveness of STFNets with extensive experiments. STFNets significantly\noutperform the state-of-the-art deep learning models in all experiments. A\nSTFNet, therefore, demonstrates superior capability as the fundamental building\nblock of deep neural networks for IoT applications for various sensor inputs.","url_abs":"http://arxiv.org/abs/1902.07849v1","url_pdf":"http://arxiv.org/pdf/1902.07849v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"stfnets-learning-sensing-signals-from-the","repo_url":"https://github.com/shuheng-li/units-sensory-time-series-classification","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"speech-recognition","task_name":"Speech Recognition"},{"task_slug":"speech-recognition-1","task_name":"speech-recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1902.07849","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}