{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/a-global-context-mechanism-for-sequence","title":"Supplementary Features of BiLSTM for Enhanced Sequence Labeling","arxiv_id":"2305.19928","date":"2023-05-31","proceeding":null,"authors":["Conglei Xu","Kun Shen","Hongguang Sun"],"abstract":"Sequence labeling tasks require the computation of sentence representations for each word within a given sentence. A prevalent method incorporates a Bi-directional Long Short-Term Memory (BiLSTM) layer to enhance the sequence structure information. However, empirical evidence Li (2020) suggests that the capacity of BiLSTM to produce sentence representations for sequence labeling tasks is inherently limited. This limitation primarily results from the integration of fragments from past and future sentence representations to formulate a complete sentence representation. In this study, we observed that the entire sentence representation, found in both the first and last cells of BiLSTM, can supplement each the individual sentence representation of each cell. Accordingly, we devised a global context mechanism to integrate entire future and past sentence representations into each cell's sentence representation within the BiLSTM framework. By incorporating the BERT model within BiLSTM as a demonstration, and conducting exhaustive experiments on nine datasets for sequence labeling tasks, including named entity recognition (NER), part of speech (POS) tagging, and End-to-End Aspect-Based sentiment analysis (E2E-ABSA). We noted significant improvements in F1 scores and accuracy across all examined datasets.","url_abs":"https://arxiv.org/abs/2305.19928v4","url_pdf":"https://arxiv.org/pdf/2305.19928v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"a-global-context-mechanism-for-sequence","repo_url":"https://github.com/conglei2xu/global-context-mechanism","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"aspect-based-sentiment-analysis-1","task_name":"Aspect-Based Sentiment Analysis"},{"task_slug":"chinese-named-entity-recognition","task_name":"Chinese Named Entity Recognition"},{"task_slug":"cg","task_name":"NER"},{"task_slug":"named-entity-recognition-ner","task_name":"Named Entity Recognition (NER)"},{"task_slug":"pos","task_name":"POS"},{"task_slug":"pos-tagging","task_name":"POS Tagging"},{"task_slug":"part-of-speech-tagging","task_name":"Part-Of-Speech Tagging"},{"task_slug":"sentence","task_name":"Sentence"},{"task_slug":"sentiment-analysis","task_name":"Sentiment Analysis"},{"task_slug":"named-entity-recognition","task_name":"named-entity-recognition"}],"methods":[{"method_slug":"attention","method_name":"Attention"},{"method_slug":"bert","method_name":"BERT"},{"method_slug":"bilstm","method_name":"BiLSTM"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"linear-warmup-with-linear-decay","method_name":"Linear Warmup With Linear Decay"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"},{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/chinese-named-entity-recognition-on-weibo-ner","task":"Chinese Named Entity Recognition","dataset":"Weibo NER","model":"BERT-BiLSTM-context","rank_in_archive_order":3,"of":18,"metrics":{"F1":"72.08"},"uses_additional_data":false},{"leaderboard":"/sota/named-entity-recognition-on-wnut-2017","task":"Named Entity Recognition (NER)","dataset":"WNUT 2017","model":"RoBERTa-BiLSTM-context","rank_in_archive_order":4,"of":23,"metrics":{"F1":"59.61"},"uses_additional_data":false}],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}