Chinese Word Segmentation

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

6 leaderboard tables shown for this task, 6 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

3 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Parent tasks archive 2025-07-28

Most implemented papers archive 2025-07-28

30 shown of 50 papers with code (244 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• ZEN: Pre-training Chinese Text Encoder Enhanced by N-gram Representations
  2 Nov 2019 7 repositories listed Syntology ran 7 of 35 samples · 28 unverified
  Moreover, it is shown that reasonable performance can be obtained when ZEN is trained on a small corpus, which is important for applying pre-training techniques to scenarios with limited data.
• PKUSEG: A Toolkit for Multi-Domain Chinese Word Segmentation
  27 Jun 2019 4 repositories listed
  Through this method, we generate synthetic data using a large amount of unlabeled data in the target domain and then obtain a word segmentation model for the target domain.
• Simplifying Neural Machine Translation with Addition-Subtraction Twin-Gated Recurrent Networks
  30 Oct 2018 3 repositories listed
  Experiments on WMT14 translation tasks demonstrate that ATR-based neural machine translation can yield competitive performance on English- German and English-French language pairs in terms of both translation quality…
• LATTE: Lattice ATTentive Encoding for Character-based Word Segmentation
  1 Jun 2023 2 repositories listed
  Our model employs the lattice structure to handle segmentation alternatives and utilizes graph neural networks along with an attention mechanism to attentively extract multi-granularity representation from the lattice…
• Unsupervised Boundary-Aware Language Model Pretraining for Chinese Sequence Labeling
  27 Oct 2022 2 repositories listed
  We apply BABERT for feature induction of Chinese sequence labeling tasks.
• Sub-Character Tokenization for Chinese Pretrained Language Models
  1 Jun 2021 2 repositories listed Syntology ran 3 of 4 samples · 1 unverified · 4 pointer-only (licence)
  2) Pronunciation-based SubChar tokenizers can encode Chinese homophones into the same transliteration sequences and produce the same tokenization output, hence being robust to homophone typos.
• Investigating Self-Attention Network for Chinese Word Segmentation
  26 Jul 2019 2 repositories listed
  Neural network has become the dominant method for Chinese word segmentation.
• Glyce: Glyph-vectors for Chinese Character Representations
  29 Jan 2019 2 repositories listed Syntology ran 3 of 8 samples · 5 unverified
  However, due to the lack of rich pictographic evidence in glyphs and the weak generalization ability of standard computer vision models on character data, an effective way to utilize the glyph information remains to be…
• LSICC: A Large Scale Informal Chinese Corpus
  26 Nov 2018 2 repositories listed
  Deep learning based natural language processing model is proven powerful, but need large-scale dataset.
• Segmental Recurrent Neural Networks
  18 Nov 2015 2 repositories listed
  Representations of the input segments (i.
• Mining Word Boundaries from Speech-Text Parallel Data for Cross-domain Chinese Word Segmentation
  12 Dec 2024 1 repository listed
  Inspired by early research on exploring naturally annotated data for Chinese Word Segmentation (CWS), and also by recent research on integration of speech and text processing, this work for the first time proposes to…
• Legal Documents Drafting with Fine-Tuned Pre-Trained Large Language Model
  6 Jun 2024 1 repository listed
  However, in the legal field application, it is difficult to obtain a large number of manually annotated data sets, which restricts the typical method applied to the task of drafting legal documents.
• The Uncertainty-based Retrieval Framework for Ancient Chinese CWS and POS
  12 Oct 2023 1 repository listed
  Automatic analysis for modern Chinese has greatly improved the accuracy of text mining in related fields, but the study of ancient Chinese is still relatively rare.
• Ancient Chinese Word Segmentation and Part-of-Speech Tagging Using Distant Supervision
  3 Mar 2023 1 repository listed
  To address this problem, we take advantage of the memorization effects of deep neural networks and a small amount of annotated data to get a model with much knowledge and a little noise, and then we use this model to…
• GNN-SL: Sequence Labeling Based on Nearest Examples via GNN
  5 Dec 2022 1 repository listed
  To better handle long-tail cases in the sequence labeling (SL) task, in this work, we introduce graph neural networks sequence labeling (GNN-SL), which augments the vanilla SL model output with similar tagging examples…
• ChiMST: A Chinese Medical Corpus for Word Segmentation and Medical Term Recognition
  1 Jun 2022 1 repository listed
  Chinese word segmentation (CWS) and named entity recognition (NER) are two important tasks in Chinese natural language processing.
• Weighted self Distillation for Chinese word segmentation
  1 May 2022 1 repository listed
  Recent researches show that multi-criteria resources and n-gram features are beneficial to Chinese Word Segmentation (CWS).
• Word Segmentation by Separation Inference for East Asian Languages
  1 May 2022 1 repository listed
  Thinking in reverse, CWS can also be viewed as a process of grouping a sequence of characters into a sequence of words.
• Unsupervised Chinese Word Segmentation with BERT Oriented Probing and Transformation
  1 May 2022 1 repository listed
  Word Segmentation is a fundamental step for understanding Chinese language.
• Fast and Accurate End-to-End Span-based Semantic Role Labeling as Word-based Graph Parsing
  6 Dec 2021 1 repository listed
  Moreover, we propose a simple constrained Viterbi procedure to ensure the legality of the output graph according to the constraints of the SRL structure.
• More than Text: Multi-modal Chinese Word Segmentation
  1 Aug 2021 1 repository listed
  Chinese word segmentation (CWS) is undoubtedly an important basic task in natural language processing.
• Exploring Word Segmentation and Medical Concept Recognition for Chinese Medical Texts
  1 Jun 2021 1 repository listed
  Chinese word segmentation (CWS) and medical concept recognition are two fundamental tasks to process Chinese electronic medical records (EMRs) and play important roles in downstream tasks for understanding Chinese EMRs.
• Joint Chinese Word Segmentation and Part-of-speech Tagging via Multi-channel Attention of Character N-grams
  1 Dec 2020 1 repository listed
  However, their work on modeling such contextual features is limited to concatenating the features or their embeddings directly with the input embeddings without distinguishing whether the contextual features are…
• Semi-supervised URL Segmentation with Recurrent Neural Networks Pre-trained on Knowledge Graph Entities
  1 Dec 2020 1 repository listed
  Breaking domain names such as openresearch into component words open and research is important for applications like Text-to-Speech synthesis and web search.
• RethinkCWS: Is Chinese Word Segmentation a Solved Task?
  13 Nov 2020 1 repository listed
  The performance of the Chinese Word Segmentation (CWS) systems has gradually reached a plateau with the rapid development of deep neural networks, especially the successful use of large pre-trained models.
• Semi-supervised URL Segmentation with Recurrent Neural NetworksPre-trained on Knowledge Graph Entities
  5 Nov 2020 1 repository listed
  Breaking domain names such as openresearch into component words open and research is important for applications like Text-to-Speech synthesis and web search.
• N-LTP: An Open-source Neural Language Technology Platform for Chinese
  24 Sep 2020 1 repository listed
  We introduce \texttt{N-LTP}, an open-source neural language technology platform supporting six fundamental Chinese NLP tasks: {lexical analysis} (Chinese word segmentation, part-of-speech tagging, and named entity…
• fastHan: A BERT-based Multi-Task Toolkit for Chinese NLP
  18 Sep 2020 1 repository listed
  The joint-model is trained and evaluated on 13 corpora of four tasks, yielding near state-of-the-art (SOTA) performance in dependency parsing and NER, achieving SOTA performance in CWS and POS.
• Approaching Neural Chinese Word Segmentation as a Low-Resource Machine Translation Task
  12 Aug 2020 1 repository listed
  Chinese word segmentation has entered the deep learning era which greatly reduces the hassle of feature engineering.
• Coupling Distant Annotation and Adversarial Training for Cross-Domain Chinese Word Segmentation
  16 Jul 2020 1 repository listed
  In order to simultaneously alleviate these two issues, this paper proposes to couple distant annotation and adversarial training for cross-domain CWS.

Syntology lines on 3 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.