{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/empath-mediapipe-aided-ensemble-learning-with","title":"EMPATH: MediaPipe-Aided Ensemble Learning with Attention-Based Transformers for Accurate Recognition of Bangla Word-Level Sign Language","arxiv_id":null,"date":"2024-12-04","proceeding":"International Conference on Pattern Recognition 2024 12","authors":["Kazi Reyazul Hasan","Muhammad Abdullah Adnan"],"abstract":"In this paper, we introduce EMPATH, an advanced computational framework developed to substantially enhance the recognition of Bangla Sign Language (BdSL). By integrating Ensemble Learning, MediaPipe Holistic for gesture tracking, and an Attention-based Transformer model, EMPATH addresses the challenges of sign language interpretation, setting new accuracy benchmarks and significantly surpassing previous records: achieving 79.81% on SignBD-Word-90 (previously best at 66.05%), 70.58% on SignBD-Word (previously best at 57%), and 99.25% on BdSL40 (previously best at 89%).\r\n\r\nA pioneering feature of EMPATH is its innovative interpolation model, built to overcome the limitations posed by missing Hand keypoints of MediaPipe. Validated in both EMPATH and a basic MLP framework, this model showcases remarkable versatility across architectures. EMPATH strategically selects its preprocessing and postprocessing techniques, optimizing each for maximum impact on accuracy and performance.\r\n\r\nExtensively trained across various word-level datasets beyond Bangla, including INCLUDE-50, INCLUDE, WLASL-100, and the Malaysian Sign Language Medical Dataset, EMPATH demonstrates its broad applicability and global potential. This diverse training scheme establishes superior accuracy benchmarks: achieving an impressive 100% on INCLUDE-50, 94.67% on INCLUDE, and 93.46% on the MSL Medical Dataset.\r\n\r\nThrough EMPATH, we aspire to bridge communication barriers for the deaf and hard-of-hearing communities, showcasing the profound impact of integrating advanced technological solutions to tackle the complexities of sign language recognition.\r\n\r\nSource code is available at https://github.com/kreyazulh/EMPATH.","url_abs":"https://link.springer.com/chapter/10.1007/978-3-031-78305-0_23","url_pdf":"https://link.springer.com/chapter/10.1007/978-3-031-78305-0_23","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"empath-mediapipe-aided-ensemble-learning-with","repo_url":"https://github.com/kreyazulh/EMPATH","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"ensemble-learning","task_name":"Ensemble Learning"},{"task_slug":"sign-language-recognition","task_name":"Sign Language Recognition"}],"methods":[{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}