{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/annotating-the-tweebank-corpus-on-named","title":"Annotating the Tweebank Corpus on Named Entity Recognition and Building NLP Models for Social Media Analysis","arxiv_id":"2201.07281","date":"2022-01-18","proceeding":"LREC 2022 6","authors":["Hang Jiang","Yining Hua","Doug Beeferman","Deb Roy"],"abstract":"Social media data such as Twitter messages (\"tweets\") pose a particular challenge to NLP systems because of their short, noisy, and colloquial nature. Tasks such as Named Entity Recognition (NER) and syntactic parsing require highly domain-matched training data for good performance. To date, there is no complete training corpus for both NER and syntactic analysis (e.g., part of speech tagging, dependency parsing) of tweets. While there are some publicly available annotated NLP datasets of tweets, they are only designed for individual tasks. In this study, we aim to create Tweebank-NER, an English NER corpus based on Tweebank V2 (TB2), train state-of-the-art (SOTA) Tweet NLP models on TB2, and release an NLP pipeline called Twitter-Stanza. We annotate named entities in TB2 using Amazon Mechanical Turk and measure the quality of our annotations. We train the Stanza pipeline on TB2 and compare with alternative NLP frameworks (e.g., FLAIR, spaCy) and transformer-based models. The Stanza tokenizer and lemmatizer achieve SOTA performance on TB2, while the Stanza NER tagger, part-of-speech (POS) tagger, and dependency parser achieve competitive performance against non-transformer models. The transformer-based models establish a strong baseline in Tweebank-NER and achieve the new SOTA performance in POS tagging and dependency parsing on TB2. We release the dataset and make both the Stanza pipeline and BERTweet-based models available \"off-the-shelf\" for use in future Tweet NLP research. Our source code, data, and pre-trained models are available at: \\url{https://github.com/social-machines/TweebankNLP}.","url_abs":"https://arxiv.org/abs/2201.07281v2","url_pdf":"https://arxiv.org/pdf/2201.07281v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"annotating-the-tweebank-corpus-on-named","repo_url":"https://github.com/social-machines/tweebanknlp","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"dependency-parsing","task_name":"Dependency Parsing"},{"task_slug":"cg","task_name":"NER"},{"task_slug":"named-entity-recognition-1","task_name":"Named Entity Recognition"},{"task_slug":"named-entity-recognition-ner","task_name":"Named Entity Recognition (NER)"},{"task_slug":"pos","task_name":"POS"},{"task_slug":"pos-tagging","task_name":"POS Tagging"},{"task_slug":"part-of-speech-tagging","task_name":"Part-Of-Speech Tagging"},{"task_slug":"named-entity-recognition","task_name":"named-entity-recognition"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/dependency-parsing-on-tweebank","task":"Dependency Parsing","dataset":"Tweebank","model":"spaCy-XLM-RoBERTa","rank_in_archive_order":3,"of":3,"metrics":{"Labelled Attachment Score":"79.39","Unlabeled Attachment Score":"83.82"},"uses_additional_data":false}],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2201.07281","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2201.07281"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/social-machines/tweebanknlp","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":4,"ran_honours":1,"unverified":10},"by_repo_kind":{"official":{"samples":15,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"724dca7b60d7f54e","entry":"build_indexes","repo":"social-machines/tweebanknlp","repo_kind":"official","path":"twitter-stanza/stanza/models/lang_identifier.py","file_url":"https://github.com/social-machines/tweebanknlp/blob/HEAD/twitter-stanza/stanza/models/lang_identifier.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"724dca7b60d7f54e"}},{"code_sha256_prefix":"03c4d0722a83c25a","entry":"convert_fc_shapes","repo":"social-machines/tweebanknlp","repo_kind":"official","path":"twitter-stanza/stanza/models/classifier.py","file_url":"https://github.com/social-machines/tweebanknlp/blob/HEAD/twitter-stanza/stanza/models/classifier.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"03c4d0722a83c25a"}},{"code_sha256_prefix":"380cf031e8cd6eec","entry":"get_batch","repo":"social-machines/tweebanknlp","repo_kind":"official","path":"twitter-stanza/stanza/models/charlm.py","file_url":"https://github.com/social-machines/tweebanknlp/blob/HEAD/twitter-stanza/stanza/models/charlm.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"380cf031e8cd6eec"}},{"code_sha256_prefix":"957a14f01a8bab3b","entry":"randomize_lengths_range","repo":"social-machines/tweebanknlp","repo_kind":"official","path":"twitter-stanza/stanza/models/lang_identifier.py","file_url":"https://github.com/social-machines/tweebanknlp/blob/HEAD/twitter-stanza/stanza/models/lang_identifier.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"957a14f01a8bab3b"}},{"code_sha256_prefix":"4372bb4533fd1936","entry":"repackage_hidden","repo":"social-machines/tweebanknlp","repo_kind":"official","path":"twitter-stanza/stanza/models/charlm.py","file_url":"https://github.com/social-machines/tweebanknlp/blob/HEAD/twitter-stanza/stanza/models/charlm.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"4372bb4533fd1936"}},{"code_sha256_prefix":"61f19438f05e0d95","entry":"batchify","repo":"social-machines/tweebanknlp","repo_kind":"official","path":"twitter-stanza/stanza/models/charlm.py","file_url":"https://github.com/social-machines/tweebanknlp/blob/HEAD/twitter-stanza/stanza/models/charlm.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"61f19438f05e0d95"}},{"code_sha256_prefix":"df9dfa89c5bddd5e","entry":"dataset_labels","repo":"social-machines/tweebanknlp","repo_kind":"official","path":"twitter-stanza/stanza/models/classifier.py","file_url":"https://github.com/social-machines/tweebanknlp/blob/HEAD/twitter-stanza/stanza/models/classifier.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"df9dfa89c5bddd5e"}},{"code_sha256_prefix":"3ad2f5b31b9f1ae3","entry":"model_file_name","repo":"social-machines/tweebanknlp","repo_kind":"official","path":"twitter-stanza/stanza/models/parser.py","file_url":"https://github.com/social-machines/tweebanknlp/blob/HEAD/twitter-stanza/stanza/models/parser.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3ad2f5b31b9f1ae3"}},{"code_sha256_prefix":"b98bb34c8f23c80a","entry":"model_file_name","repo":"social-machines/tweebanknlp","repo_kind":"official","path":"twitter-stanza/stanza/models/tagger.py","file_url":"https://github.com/social-machines/tweebanknlp/blob/HEAD/twitter-stanza/stanza/models/tagger.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b98bb34c8f23c80a"}},{"code_sha256_prefix":"e6d73d86addb1bf5","entry":"parse_args","repo":"social-machines/tweebanknlp","repo_kind":"official","path":"twitter-stanza/stanza/models/identity_lemmatizer.py","file_url":"https://github.com/social-machines/tweebanknlp/blob/HEAD/twitter-stanza/stanza/models/identity_lemmatizer.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e6d73d86addb1bf5"}},{"code_sha256_prefix":"ed203c36edd90a76","entry":"parse_args","repo":"social-machines/tweebanknlp","repo_kind":"official","path":"twitter-stanza/stanza/models/lang_identifier.py","file_url":"https://github.com/social-machines/tweebanknlp/blob/HEAD/twitter-stanza/stanza/models/lang_identifier.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ed203c36edd90a76"}},{"code_sha256_prefix":"1377f605ed321af1","entry":"parse_args","repo":"social-machines/tweebanknlp","repo_kind":"official","path":"twitter-stanza/stanza/models/mwt_expander.py","file_url":"https://github.com/social-machines/tweebanknlp/blob/HEAD/twitter-stanza/stanza/models/mwt_expander.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1377f605ed321af1"}},{"code_sha256_prefix":"e204f019f1d98b1a","entry":"parse_args","repo":"social-machines/tweebanknlp","repo_kind":"official","path":"twitter-stanza/stanza/models/ner_tagger.py","file_url":"https://github.com/social-machines/tweebanknlp/blob/HEAD/twitter-stanza/stanza/models/ner_tagger.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"e204f019f1d98b1a"}},{"code_sha256_prefix":"955a84f0cd791e45","entry":"parse_args","repo":"social-machines/tweebanknlp","repo_kind":"official","path":"twitter-stanza/stanza/models/parser.py","file_url":"https://github.com/social-machines/tweebanknlp/blob/HEAD/twitter-stanza/stanza/models/parser.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"955a84f0cd791e45"}},{"code_sha256_prefix":"3f3eeedc7dc1038c","entry":"parse_args","repo":"social-machines/tweebanknlp","repo_kind":"official","path":"twitter-stanza/stanza/models/tagger.py","file_url":"https://github.com/social-machines/tweebanknlp/blob/HEAD/twitter-stanza/stanza/models/tagger.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3f3eeedc7dc1038c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}