Papers › Semi-Supervised Sequence Modeling with Cross-View Training
Semi-Supervised Sequence Modeling with Cross-View Training
Kevin Clark, Minh-Thang Luong, Christopher D. Manning, Quoc V. Le
Unsupervised representation learning algorithms such as word2vec and ELMo improve the accuracy of many supervised NLP models, mainly because they can take advantage of large amounts of unlabeled text. However, the supervised models only learn from task-specific labeled data during the main training phase. We therefore propose Cross-View Training (CVT), a semi-supervised learning algorithm that improves the representations of a Bi-LSTM sentence encoder using a mix of labeled and unlabeled data. On labeled examples, standard supervised learning is used. On unlabeled examples, CVT teaches auxiliary prediction modules that see restricted views of the input (e.g., only part of a sentence) to match the predictions of the full model seeing the whole input. Since the auxiliary modules and the full model share intermediate representations, this in turn improves the full model. Moreover, we show that CVT is particularly effective when combined with multi-task learning. We evaluate CVT on five sequence tagging tasks, machine translation, and dependency parsing, achieving state-of-the-art results.
In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| CCG Supertagging | CCGbank | CVT + Multi-task + Large | Accuracy | 96.1 | #3 of 8 | Archive leaderboard | report |
| Dependency Parsing | Penn Treebank | CVT + Multi-Task | LAS | 95.02 | #7 of 22 | Archive leaderboard | report |
| Dependency Parsing | Penn Treebank | CVT + Multi-Task | UAS | 96.61 | #7 of 22 | Archive leaderboard | report |
| Machine Translation | IWSLT2015 English-Vietnamese | CVT | BLEU | 29.6 | #6 of 11 | Archive leaderboard | report |
| Named Entity Recognition (NER) | CoNLL 2003 (English) | CVT + Multi-Task | F1 | 92.61 | #39 of 73 | Archive leaderboard | report |
| Named Entity Recognition (NER) | CoNLL 2003 (English) | CVT + Multi-Task + Large | F1 | 92.61 | #40 of 73 | Archive leaderboard | report |
| Named Entity Recognition (NER) | Ontonotes v5 (English) | CVT + Multi-Task + Large | F1 | 88.81 | #18 of 28 | Archive leaderboard | report |
| Part-Of-Speech Tagging | Penn Treebank | CVT + Multi-task | Accuracy | 97.76 | #5 of 20 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections