Methods › Natural Language Processing › Subword Segmentation › Gradient-Based Subword Tokenization
GBST
Gradient-Based Subword Tokenization
Introduced by Yi Tay et al. in Charformer: Fast Character Transformers via Gradient-based Subword Tokenization
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
GBST, or Gradient-based Subword Tokenization Module, is a soft gradient-based subword tokenization module that automatically learns latent subword representations from characters in a data-driven fashion. Concretely, GBST enumerates candidate subword blocks and learns to score them in a position-wise fashion using a block scoring network.
GBST learns a position-wise soft selection over candidate subword blocks by scoring them with a scoring network. In contrast to prior tokenization-free methods, GBST learns interpretable latent subwords, which enables easy inspection of lexical representations and is more efficient than other byte-based models.
Papers archive 2025-07-28
5 shown of 5, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
CharFormer: A Glyph Fusion based Attentive Framework for High-precision Character Image Denoising 16 Jul 2022 · 1 repository · arXiv:2207.07798
-
Patching Leaks in the Charformer for Efficient Character-Level Generation 27 May 2022 · 1 repository · arXiv:2205.14086
-
A New Generation of Perspective API: Efficient Multilingual Character-level Transformers 22 Feb 2022 · 0 repositories · arXiv:2202.11176
-
Patching Leaks in the Charformer for Generative Tasks 16 Jan 2022 · 0 repositories
-
Charformer: Fast Character Transformers via Gradient-based Subword Tokenization 23 Jun 2021 · 2 repositories · arXiv:2106.12672Syntology ran 7 of 10 samples · 3 unverified
Tasks archive 2025-07-28
12 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections