{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/does-higher-order-lstm-have-better-accuracy","title":"Does Higher Order LSTM Have Better Accuracy for Segmenting and Labeling Sequence Data?","arxiv_id":"1711.08231","date":"2017-11-22","proceeding":"COLING 2018 8","authors":["Yi Zhang","Xu sun","Shuming Ma","Yang Yang","Xuancheng Ren"],"abstract":"Existing neural models usually predict the tag of the current token\nindependent of the neighboring tags. The popular LSTM-CRF model considers the\ntag dependencies between every two consecutive tags. However, it is hard for\nexisting neural models to take longer distance dependencies of tags into\nconsideration. The scalability is mainly limited by the complex model\nstructures and the cost of dynamic programming during training. In our work, we\nfirst design a new model called \"high order LSTM\" to predict multiple tags for\nthe current token which contains not only the current tag but also the previous\nseveral tags. We call the number of tags in one prediction as \"order\". Then we\npropose a new method called Multi-Order BiLSTM (MO-BiLSTM) which combines low\norder and high order LSTMs together. MO-BiLSTM keeps the scalability to high\norder models with a pruning technique. We evaluate MO-BiLSTM on all-phrase\nchunking and NER datasets. Experiment results show that MO-BiLSTM achieves the\nstate-of-the-art result in chunking and highly competitive results in two NER\ndatasets.","url_abs":"http://arxiv.org/abs/1711.08231v3","url_pdf":"http://arxiv.org/pdf/1711.08231v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"does-higher-order-lstm-have-better-accuracy","repo_url":"https://github.com/lancopku/Multi-Order-LSTM","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":null}],"tasks":[{"task_slug":"chunking","task_name":"Chunking"},{"task_slug":"cg","task_name":"NER"},{"task_slug":"tag","task_name":"TAG"}],"methods":[{"method_slug":"bilstm","method_name":"BiLSTM"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"pruning","method_name":"Pruning"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}