Browse State-of-the-Art › Chinese Word Segmentation
Chinese Word Segmentation
50 papers with code · 6 benchmarks · 3 datasets archive 2025-07-28
Chinese word segmentation is the task of splitting Chinese text (i.e. a sequence of Chinese characters) into words (Source: www.nlpprogress.com).
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
6 leaderboard tables shown for this task, 6 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.
Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. Each line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the row's paper, the line starts with that repository's state (“official” is the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from. A dash means no Syntology record for that paper, not a recorded non-run. Each “ran” line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
3 datasets whose archive record lists this task, ordered by the archive's paper count.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Parent tasks archive 2025-07-28
Papers archive 2025-07-28
archive papers tagged: 244 · with a code link: 50 · where Syntology ran a sample: 3 (2 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (3 of 244 tagged: 2 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 3: papers 1 to 100 of 244, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
2 Nov 2019 7 repositories listed Syntology official (archive's flag): 1 ran · 28 ran (of which 0 constructed an object rather than computing a result; 22 with no instrument failure: 1 honoured, 0 violated, 21 with no contract checked; 6 where Syntology's instrument failed) · 7 unverified (of 35 harvested samples) · 2 pointer-only (licence)Moreover, it is shown that reasonable performance can be obtained when ZEN is trained on a small corpus, which is important for applying pre-training techniques to scenarios with limited data.
-
27 Jun 2019 4 repositories listedThrough this method, we generate synthetic data using a large amount of unlabeled data in the target domain and then obtain a word segmentation model for the target domain.
-
30 Oct 2018 3 repositories listedExperiments on WMT14 translation tasks demonstrate that ATR-based neural machine translation can yield competitive performance on English- German and English-French language pairs in terms of both translation quality…
-
1 Jun 2023 2 repositories listedOur model employs the lattice structure to handle segmentation alternatives and utilizes graph neural networks along with an attention mechanism to attentively extract multi-granularity representation from the lattice…
-
27 Oct 2022 2 repositories listedWe apply BABERT for feature induction of Chinese sequence labeling tasks.
-
1 Jun 2021 2 repositories listed Syntology official (archive's flag): 2 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 4 harvested samples) · 4 pointer-only (licence)2) Pronunciation-based SubChar tokenizers can encode Chinese homophones into the same transliteration sequences and produce the same tokenization output, hence being robust to homophone typos.
-
26 Jul 2019 2 repositories listedNeural network has become the dominant method for Chinese word segmentation.
-
29 Jan 2019 2 repositories listed Syntology official (archive's flag): 8 ran · 8 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 2 honoured, 0 violated, 5 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 8 harvested samples) · 1 pointer-only (licence)However, due to the lack of rich pictographic evidence in glyphs and the weak generalization ability of standard computer vision models on character data, an effective way to utilize the glyph information remains to be…
-
26 Nov 2018 2 repositories listedDeep learning based natural language processing model is proven powerful, but need large-scale dataset.
-
18 Nov 2015 2 repositories listedRepresentations of the input segments (i. e., contiguous subsequences of the input) are computed by encoding their constituent tokens using bidirectional recurrent neural nets, and these "segment embeddings" are used to…
-
12 Dec 2024 1 repository listedInspired by early research on exploring naturally annotated data for Chinese Word Segmentation (CWS), and also by recent research on integration of speech and text processing, this work for the first time proposes to…
-
6 Jun 2024 1 repository listedHowever, in the legal field application, it is difficult to obtain a large number of manually annotated data sets, which restricts the typical method applied to the task of drafting legal documents.
-
12 Oct 2023 1 repository listedAutomatic analysis for modern Chinese has greatly improved the accuracy of text mining in related fields, but the study of ancient Chinese is still relatively rare.
-
3 Mar 2023 1 repository listedTo address this problem, we take advantage of the memorization effects of deep neural networks and a small amount of annotated data to get a model with much knowledge and a little noise, and then we use this model to…
-
5 Dec 2022 1 repository listedTo better handle long-tail cases in the sequence labeling (SL) task, in this work, we introduce graph neural networks sequence labeling (GNN-SL), which augments the vanilla SL model output with similar tagging examples…
-
1 Jun 2022 1 repository listedChinese word segmentation (CWS) and named entity recognition (NER) are two important tasks in Chinese natural language processing.
-
1 May 2022 1 repository listedWord Segmentation is a fundamental step for understanding Chinese language.
-
1 May 2022 1 repository listedRecent researches show that multi-criteria resources and n-gram features are beneficial to Chinese Word Segmentation (CWS).
-
1 May 2022 1 repository listedThinking in reverse, CWS can also be viewed as a process of grouping a sequence of characters into a sequence of words.
-
6 Dec 2021 1 repository listedMoreover, we propose a simple constrained Viterbi procedure to ensure the legality of the output graph according to the constraints of the SRL structure.
-
1 Aug 2021 1 repository listedChinese word segmentation (CWS) is undoubtedly an important basic task in natural language processing.
-
1 Jun 2021 1 repository listedChinese word segmentation (CWS) and medical concept recognition are two fundamental tasks to process Chinese electronic medical records (EMRs) and play important roles in downstream tasks for understanding Chinese EMRs.
-
1 Dec 2020 1 repository listedHowever, their work on modeling such contextual features is limited to concatenating the features or their embeddings directly with the input embeddings without distinguishing whether the contextual features are…
-
1 Dec 2020 1 repository listedBreaking domain names such as openresearch into component words open and research is important for applications like Text-to-Speech synthesis and web search.
-
13 Nov 2020 1 repository listedThe performance of the Chinese Word Segmentation (CWS) systems has gradually reached a plateau with the rapid development of deep neural networks, especially the successful use of large pre-trained models.
-
5 Nov 2020 1 repository listedBreaking domain names such as openresearch into component words open and research is important for applications like Text-to-Speech synthesis and web search.
-
24 Sep 2020 1 repository listedWe introduce N-LTP, an open-source neural language technology platform supporting six fundamental Chinese NLP tasks: {lexical analysis} (Chinese word segmentation, part-of-speech tagging, and named entity recognition)…
-
18 Sep 2020 1 repository listedThe joint-model is trained and evaluated on 13 corpora of four tasks, yielding near state-of-the-art (SOTA) performance in dependency parsing and NER, achieving SOTA performance in CWS and POS.
-
12 Aug 2020 1 repository listedChinese word segmentation has entered the deep learning era which greatly reduces the hassle of feature engineering.
-
16 Jul 2020 1 repository listedIn order to simultaneously alleviate these two issues, this paper proposes to couple distant annotation and adversarial training for cross-domain CWS.
-
1 Jul 2020 1 repository listed
-
1 Jul 2020 1 repository listed
-
29 Nov 2019 1 repository listed
-
31 Oct 2019 1 repository listed
-
28 Jun 2019 1 repository listed
-
9 Apr 2019 1 repository listed
-
5 Mar 2019 1 repository listed
-
30 Oct 2018 1 repository listed
-
7 Oct 2018 1 repository listed
-
1 Oct 2018 1 repository listed
-
20 Aug 2018 1 repository listed
-
1 Aug 2018 1 repository listed
-
23 Dec 2017 1 repository listed
-
7 Dec 2017 1 repository listed
-
13 Nov 2017 1 repository listed
-
24 Apr 2017 1 repository listed
-
1 Aug 2016 1 repository listed
-
14 Jun 2016 1 repository listed
-
1 Jun 2016 1 repository listed
-
19 Apr 2016 1 repository listed
-
Parsing Through Boundaries in Chinese Word Segmentation29 Mar 2025 0 repositories listed
-
An Effective Incorporating Heterogeneous Knowledge Curriculum Learning for Sequence Labeling21 Feb 2024 0 repositories listed
-
Incorporating Deep Syntactic and Semantic Knowledge for Chinese Sequence Labeling with GCN3 Jun 2023 0 repositories listed
-
Joint Chinese Word Segmentation and Span-based Constituency Parsing3 Nov 2022 0 repositories listed
-
Multifaceted Assessments of Traditional Chinese Word Segmentation Tool on Large Corpora1 Nov 2022 0 repositories listed
-
Mining Word Boundaries in Speech as Naturally Annotated Word Segmentation Data31 Oct 2022 0 repositories listed
-
基于神经网络的半监督CRF中文分词(Semi-supervised CRF Chinese Word Segmentation based on Neural Network)1 Oct 2022 0 repositories listed
-
That Slepen Al the Nyght with Open Ye! Cross-era Sequence Segmentation with Switch-memory7 Sep 2022 0 repositories listed
-
A Joint Framework for Ancient Chinese WS and POS Tagging Based on Adversarial Ensemble Learning1 Jun 2022 0 repositories listed
-
Ancient Chinese Word Segmentation and Part-of-Speech Tagging Using Data Augmentation1 Jun 2022 0 repositories listed
-
BERT 4EVER@EvaHan 2022: Ancient Chinese Word Segmentation and Part-of-Speech Tagging Based on Adversarial Learning and Continual Pre-training1 Jun 2022 0 repositories listed
-
The First International Ancient Chinese Word Segmentation and POS Tagging Bakeoff: Overview of the EvaHan 2022 Evaluation Campaign1 Jun 2022 0 repositories listed
-
A New Evaluation Method: Evaluation Data and Metrics for Chinese Grammar Error Correction30 Apr 2022 0 repositories listed
-
Chinese Word Segmentation with Heterogeneous Graph Neural Network22 Jan 2022 0 repositories listed
-
Joint Chinese Word Segmentation and Part-of-speech Tagging via Two-stage Span Labeling17 Dec 2021 0 repositories listed
-
Green CWS: Extreme Distillation and Efficient Decode Method Towards Industrial Application17 Nov 2021 0 repositories listed
-
That Slepen Al the Nyght with Open Ye! Cross-era Sequence Segmentation with Switch-memory16 Nov 2021 0 repositories listed
-
Unsupervised Chinese Word Segmentation with BERT Oriented Probing and Transformation16 Nov 2021 0 repositories listed
-
Improving Prosody for Unseen Texts in Speech Synthesis by Utilizing Linguistic Information and Noisy Data15 Nov 2021 0 repositories listed
-
Lexicon-Based Graph Convolutional Network for Chinese Word Segmentation1 Nov 2021 0 repositories listed
-
Segment, Mask, and Predict: Augmenting Chinese Word Segmentation with Self-Supervision1 Nov 2021 0 repositories listed
-
Span Labeling Approach for Vietnamese and Chinese Word Segmentation1 Oct 2021 0 repositories listed
-
Enhancing Chinese Word Segmentation via Pseudo Labels for Practicability1 Aug 2021 0 repositories listed
-
Federated Chinese Word Segmentation with Global Character Associations1 Aug 2021 0 repositories listed
-
基于双编码器的医学文本中文分词(Chinese word segmentation of medical text based on dual-encoder)1 Aug 2021 0 repositories listed
-
MVP-BERT: Multi-Vocab Pre-training for Chinese BERT1 Aug 2021 0 repositories listed
-
Bidirectional LSTM-CRF Attention-based Model for Chinese Word Segmentation20 May 2021 0 repositories listed
-
A More Efficient Chinese Named Entity Recognition base on BERT and Syntactic Analysis11 Jan 2021 0 repositories listed
-
Segmenting Natural Language Sentences via Lexical Unit Analysis10 Dec 2020 0 repositories listed
-
Context-Aware Word Segmentation for Chinese Real-World Discourse1 Dec 2020 0 repositories listed
-
Multi-grained Chinese Word Segmentation with Weakly Labeled Data1 Dec 2020 0 repositories listed
-
Towards Fast and Accurate Neural Chinese Word Segmentation with Multi-Criteria Learning1 Dec 2020 0 repositories listed
-
MVP-BERT: Redesigning Vocabularies for Chinese BERT and Multi-Vocab Pretraining17 Nov 2020 0 repositories listed
-
A Joint Multiple Criteria Model in Transfer Learning for Cross-domain Chinese Word Segmentation1 Nov 2020 0 repositories listed
-
Pre-training with Meta Learning for Chinese Word Segmentation23 Oct 2020 0 repositories listed
-
A Joint Model for Graph-based Chinese Dependency Parsing1 Oct 2020 0 repositories listed
-
Neural Graph Matching Networks for Chinese Short Text Matching1 Jul 2020 0 repositories listed
-
Incorporating Uncertain Segmentation Information into Chinese NER for Social Media Text14 Apr 2020 0 repositories listed
-
Unified Multi-Criteria Chinese Word Segmentation with BERT13 Apr 2020 0 repositories listed
-
Integrating Boundary Assembling into a DNN Framework for Named Entity Recognition in Chinese Social Media Text27 Feb 2020 0 repositories listed
-
A New Clustering neural network for Chinese word segmentation18 Feb 2020 0 repositories listed
-
BERT Meets Chinese Word Segmentation20 Sep 2019 0 repositories listed
-
NE-LP: Normalized Entropy and Loss Prediction based Sampling for Active Learning in Chinese Word Segmentation on EHRs22 Aug 2019 0 repositories listed
-
Shrinking Japanese Morphological Analyzers With Neural Networks and Semi-supervised Learning1 Jun 2019 0 repositories listed
-
A realistic and robust model for Chinese word segmentation21 May 2019 0 repositories listed
-
A Seq-to-Seq Transformer Premised Temporal Convolutional Network for Chinese Word Segmentation21 May 2019 0 repositories listed
-
Is Word Segmentation Necessary for Deep Learning of Chinese Representations?14 May 2019 0 repositories listed
-
Neural Chinese Word Segmentation with Lexicon and Unlabeled Data via Posterior Regularization26 Apr 2019 0 repositories listed
-
Toward Fast and Accurate Neural Chinese Word Segmentation with Multi-Criteria Learning11 Mar 2019 0 repositories listed
-
Chinese Word Segmentation: Another Decade Review (2007-2017)18 Jan 2019 0 repositories listed
Syntology lines on 3 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections