Browse State-of-the-Art › Lexical Analysis

Lexical Analysis

8 papers with code · 0 benchmarks · 1 dataset archive 2025-07-28

Natural Language Processing

Lexical analysis is the process of converting a sequence of characters into a sequence of tokens (strings with an assigned and thus identified meaning). (Source: Adapted from Wikipedia)

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

1 dataset whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

1 subtask in the archive's task tree.

Papers archive 2025-07-28

archive papers tagged: 56 · with a code link: 8 · where Syntology ran a sample: 0 (0 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument) Syntology

Show: all tagged papersonly where code ran: none of the 56 tagged papers has a Syntology run on record

Page 1 of 1: papers 1 to 56 of 56, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.

Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.

  • 4 Jun 2021 2 repositories listed
    This paper describes the performance of the team cs60075_team2 at SemEval 2021 Task 1 - Lexical Complexity Prediction.
  • 5 Jul 2018 2 repositories listed
    Lexical analysis is believed to be a crucial step towards natural language understanding and has been widely studied.
  • 3 Oct 2024 1 repository listed
    Lexical tones play a crucial role in Sino-Tibetan languages.
  • 16 Aug 2022 1 repository listed
    Through a hierarchical clustering of the data in this lower-dimensional space, this method yields clear cultural areas and the topics of discussion that define them.
  • 1 Jan 2021 1 repository listed
    The results reported in this paper aim to increase the presence of the Uzbek language in the Internet and its usability within IT applications.
  • 24 Sep 2020 1 repository listed
    We introduce N-LTP, an open-source neural language technology platform supporting six fundamental Chinese NLP tasks: {lexical analysis} (Chinese word segmentation, part-of-speech tagging, and named entity recognition)…
  • 26 Sep 2019 1 repository listed
    We apply this framework to existing datasets and models and show that: (1) the pivot words are strong features for the classification of sentence attributes; (2) to change the attribute of a sentence, many datasets only…
  • 5 Feb 2015 1 repository listed
    Based on the Aristotelian concept of potentiality vs. actuality allowing for the study of energy and dynamics in language, we propose a field approach to lexical analysis.
  • A Tale of Two Identities: An Ethical Audit of Human and AI-Crafted Personas
    7 May 2025 0 repositories listed
    As LLMs (large language models) are increasingly used to generate synthetic personas particularly in data-limited domains such as health, privacy, and HCI, it becomes necessary to understand how these narratives…
  • Polarization and Morality: Lexical Analysis of Abortion Discourse on Reddit
    29 Jun 2024 0 repositories listed
    This study investigates whether division on political topics is mapped with the distinctive patterns of language use.
  • Computational lexical analysis of Flamenco genres
    9 May 2024 0 repositories listed
    Flamenco, recognized by UNESCO as part of the Intangible Cultural Heritage of Humanity, is a profound expression of cultural identity rooted in Andalusia, Spain.
  • Towards Lexical Analysis of Dog Vocalizations via Online Videos
    21 Sep 2023 0 repositories listed
    Deciphering the semantics of animal language has been a grand challenge.
  • Leveraging Natural Language Processing For Public Health Screening On YouTube: A COVID-19 Case Study
    1 Jun 2023 0 repositories listed
    Background: Social media platforms have become a viable source of medical information, with patients and healthcare professionals using them to share health-related information and track diseases.
  • Happy or grumpy? A Machine Learning Approach to Analyze the Sentiment of Airline Passengers' Tweets
    28 Sep 2022 0 repositories listed
    As one of the most extensive social networking services, Twitter has more than 300 million active users as of 2022.
  • Analyse Automatique de l’Ancien Arménien. Évaluation d’une méthode hybride « dictionnaire » et « réseau de neurones » sur un Extrait de l’Adversus Haereses d’Irénée de Lyon
    1 Jun 2022 0 repositories listed
    The aim of this paper is to evaluate a lexical analysis (mainly lemmatization and POS-tagging) of a sample of the Ancient Armenian version of the Adversus Haereses by Irenaeus of Lyons (2nd c.)
  • CV4Code: Sourcecode Understanding via Visual Code Representations
    11 May 2022 0 repositories listed
    We present CV4Code, a compact and effective computer vision method for sourcecode understanding.
  • Effect and Analysis of Large-scale Language Model Rescoring on Competitive ASR Systems
    1 Apr 2022 0 repositories listed
    Large-scale language models (LLMs) such as GPT-2, BERT and RoBERTa have been successfully applied to ASR N-best rescoring.
  • Evaluating Persian Tokenizers
    22 Feb 2022 0 repositories listed
    Tokenization plays a significant role in the process of lexical analysis.
  • COVID-19 and Misinformation: A Large-Scale Lexical Analysis on Twitter
    1 Aug 2021 0 repositories listed
    Social media is often used by individuals and organisations as a platform to spread misinformation.
  • Visual Exploration and Knowledge Discovery from Biomedical Dark Data
    28 Sep 2020 0 repositories listed
    Data visualization techniques proffer efficient means to organize and present data in graphically appealing formats, which not only speeds up the process of decision making and pattern recognition but also enables…
  • Integration of Automatic Sentence Segmentation and Lexical Analysis of Ancient Chinese based on BiLSTM-CRF Model
    1 May 2020 0 repositories listed
    The basic tasks of ancient Chinese information processing include automatic sentence segmentation, word segmentation, part-of-speech tagging and named entity recognition.
  • Processing Language Resources of Under-Resourced and Endangered Languages for the Generation of Augmentative Alternative Communication Boards
    1 May 2020 0 repositories listed
    Under-resourced and endangered or small languages yield problems for automatic processing and exploiting because of the small amount of available data.
  • An Intelligent Data Analysis for Hotel Recommendation Systems using Machine Learning
    15 Oct 2019 0 repositories listed
    This paper presents an intelligent approach to handle heterogeneous and large-sized data using machine learning to generate true recommendations for the future customers.
  • Recognizing Topic Change in Search Sessions of Digital Libraries based on Thesaurus and Classification System
    24 Sep 2019 0 repositories listed
    Log analysis in Web search showed that user sessions often contain several different topics.
  • Corpus Lexicography in a Wider Context
    1 Sep 2019 0 repositories listed
    This paper describes a set of tools that offers comprehensive solutions for corpus lexicography.
  • LEGATO: A flexible lexicographic annotation tool
    1 Sep 2019 0 repositories listed
    This article is a report from an ongoing project aiming at analyzing lexical and grammatical competences of Swedish as a Second language (L2).
  • A Comprehensive Analysis of Twitter Trending Topics
    21 Jul 2019 0 repositories listed
    In Twitter, a name, phrase, or topic that is mentioned at a greater rate than others is called a "trending topic" or simply "trend".
  • The Truth and Nothing but the Truth: Multimodal Analysis for Deception Detection
    11 Mar 2019 0 repositories listed
    We propose a data-driven method for automatic deception detection in real-life trial data using visual and verbal cues.
  • Content Extraction and Lexical Analysis from Customer-Agent Interactions
    1 Nov 2018 0 repositories listed
    In this paper, we provide a lexical comparative analysis of the vocabulary used by customers and agents in an Enterprise Resource Planning (ERP) environment and a potential solution to clean the data and extract…
  • Multitask and Multilingual Modelling for Lexical Analysis
    7 Sep 2018 0 repositories listed
    In Natural Language Processing (NLP), one traditionally considers a single task (e.g. part-of-speech tagging) for a single language (e.g. English) at a time.
  • TRAPACC and TRAPACCS at PARSEME Shared Task 2018: Neural Transition Tagging of Verbal Multiword Expressions
    1 Aug 2018 0 repositories listed
  • Automatic Annotation of Semantic Term Types in the Complete ACL Anthology Reference Corpus
    1 May 2018 0 repositories listed
  • Towards a music-language mapping
    1 May 2018 0 repositories listed
  • One Model to Rule them all: Multitask and Multilingual Modelling for Lexical Analysis
    3 Nov 2017 0 repositories listed
  • Deep Learning in Lexical Analysis and Parsing
    1 Nov 2017 0 repositories listed
  • Benchmarking Joint Lexical and Syntactic Analysis on Multiword-Rich Data
    1 Apr 2017 0 repositories listed
  • Parsing and MWE Detection: Fips at the PARSEME Shared Task
    1 Apr 2017 0 repositories listed
  • The ATILF-LLF System for Parseme Shared Task: a Transition-based Verbal Multiword Expression Tagger
    1 Apr 2017 0 repositories listed
  • Sarcasm Detection : Building a Contextual Hierarchy
    1 Dec 2016 0 repositories listed
  • Detecting Mild Cognitive Impairment by Exploiting Linguistic Information from Transcripts
    1 Aug 2016 0 repositories listed
  • Deep Lexical Segmentation and Syntactic Parsing in the Easy-First Dependency Framework
    1 Jun 2016 0 repositories listed
  • Lifetime Achievement Award: Translating Today into Tomorrow
    1 Dec 2015 0 repositories listed
  • Ontology Based SMS Controller for Smart Phones
    4 Sep 2015 0 repositories listed
  • Lexical Characteristics Analysis of Chinese Clinical Documents
    1 Jul 2015 0 repositories listed
  • Noun Paraphrasing Based on a Variety of Contexts
    1 Dec 2014 0 repositories listed
  • Dependency Parsing: Past, Present, and Future
    1 Aug 2014 0 repositories listed
  • BACANAL: Short Length Random Walks For Lexical Analysis, Application to lexical substitution (BACANAL : Balades Aléatoires Courtes pour ANAlyses Lexicales Application à la substitution lexicale) [in French]
    1 Jul 2014 0 repositories listed
  • Augmenting FrameNet Via PPDB
    1 Jun 2014 0 repositories listed
  • Toward Macro-Insights for Suicide Prevention: Analyzing Fine-Grained Distress at Scale
    1 Jun 2014 0 repositories listed
  • Terminology localization guidelines for the national scenario
    1 May 2014 0 repositories listed
  • Corpus-driven Lexical Analysis: Norms and Exploitations in Word Use
    1 Nov 2013 0 repositories listed
  • Selecting Proper Lexical Paraphrase for Children
    1 Oct 2013 0 repositories listed
  • History Based Unsupervised Data Oriented Parsing
    1 Sep 2013 0 repositories listed
  • Toward an amazigh language processing
    1 Dec 2012 0 repositories listed
  • On using context for automatic correction of non-word misspellings in student essays
    1 Jun 2012 0 repositories listed
  • Large Scale Lexical Analysis
    1 May 2012 0 repositories listed

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections