{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/predicting-tongue-motion-in-unlabeled","title":"Predicting tongue motion in unlabeled ultrasound videos using convolutional LSTM neural network","arxiv_id":"1902.06927","date":"2019-02-19","proceeding":null,"authors":["Chaojie Zhao","Peng Zhang","Jian Zhu","Chengrui Wu","Huaimin Wang","Kele Xu"],"abstract":"A challenge in speech production research is to predict future tongue\nmovements based on a short period of past tongue movements. This study tackles\nspeaker-dependent tongue motion prediction problem in unlabeled ultrasound\nvideos with convolutional long short-term memory (ConvLSTM) networks. The model\nhas been tested on two different ultrasound corpora. ConvLSTM outperforms\n3-dimensional convolutional neural network (3DCNN) in predicting the\n9\\textsuperscript{th} frames based on 8 preceding frames, and also demonstrates\ngood capacity to predict only the tongue contours in future frames. Further\ntests reveal that ConvLSTM can also learn to predict tongue movements in more\ndistant frames beyond the immediately following frames. Our codes are available\nat: https://github.com/shuiliwanwu/ConvLstm-ultrasound-videos.","url_abs":"http://arxiv.org/abs/1902.06927v1","url_pdf":"http://arxiv.org/pdf/1902.06927v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"predicting-tongue-motion-in-unlabeled","repo_url":"https://github.com/shuiliwanwu/ConvLstm-ultrasound-videos","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"motion-prediction","task_name":"motion prediction"}],"methods":[{"method_slug":"convlstm","method_name":"ConvLSTM"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}