Methods › Natural Language Processing › Static Word Embeddings › Mirror-BERT

Mirror-BERT

6 papers tagged archive 2025-07-28

Introduced by Fangyu Liu et al. in Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders

archive 2025-07-28 Description, source and code snippet are the archive's method entry.

Mirror-BERT converts pretrained language models into effective universal text encoders without any supervision, in 20-30 seconds. It is an extremely simple, fast, and effective contrastive learning technique. It relies on fully identical or slightly modified string pairs as positive (i.e., synonymous) fine-tuning examples, and aims to maximise their similarity during identity fine-tuning.

Paper Source See Code · cambridgeltl/mirror-bert

Papers archive 2025-07-28

6 shown of 6, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.

TaCL: Improving BERT Pre-training with Token-aware Contrastive Learning
7 Nov 2021 · 2 repositories · arXiv:2111.04198
Prix-LM: Pretraining for Multilingual Knowledge Base Construction
16 Oct 2021 · 1 repository · arXiv:2110.08443
Rewire-then-Probe: A Contrastive Recipe for Probing Biomedical Knowledge of Pre-trained Language Models
15 Oct 2021 · 1 repository · arXiv:2110.08173
Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations
27 Sep 2021 · 1 repository · arXiv:2109.13059
MirrorWiC: On Eliciting Word-in-Context Representations from Pretrained Language Models
19 Sep 2021 · 1 repository · arXiv:2109.09237
Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders
16 Apr 2021 · 1 repository · arXiv:2104.08027

Syntology ran 0 of 1 samples · 1 unverified

Tasks archive 2025-07-28

20 shown of 22 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.

Task	Papers
Contrastive Learning	4
Sentence	3
Entity Linking	2
Language Modelling	2
Semantic Textual Similarity	2
Sentence Similarity	2
Bilingual Lexicon Induction	1
Causal Language Modeling	1
Contextualised Word Representations	1
Cross-Lingual Entity Linking	1
Cross-Lingual Semantic Textual Similarity	1
Knowledge Base Construction	1
Knowledge Probing	1
Language Modeling	1
Link Prediction	1
Natural Language Understanding	1
Paraphrase Identification	1
STS	1
Semantic Similarity	1
Transfer Learning	1

Usage over time archive 2025-07-28

Papers per year the archive tags with this method, by the paper's archive date (6 dated). Bars are counts, not a trend claim.

Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).

Categories archive 2025-07-28

Static Word Embeddings Contextualized Word Embeddings Word Embeddings Sentence Embeddings Self-Supervised Learning

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections