Methods › Natural Language Processing › Transformers › ELECTRA
ELECTRA
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
ELECTRA is a transformer with a new pre-training approach which trains two transformer models: the generator and the discriminator. The generator replaces tokens in the sequence - trained as a masked language model - and the discriminator (the ELECTRA contribution) attempts to identify which tokens are replaced by the generator in the sequence. This pre-training task is called replaced token detection, and is a replacement for masking the input.
In Syntology repository known, no verified implementation (declared_code; repositories: 5 listed of 20 in the graph, strongest link: official). The archive credits no paper with an arXiv id for this method, so whether this answer (about arXiv:2503.08305) is about this method is not checked.
Origin paper of this answer, arXiv:2503.08305 (its page on this site · its page on Syntology): Syntology's run record for its harvested code, 8 ran (of which 0 constructed an object rather than computing a result; 8 with no instrument failure: 1 honoured, 0 violated, 7 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 9 harvested samples). The instrument figure counts failures of Syntology's instrument, not of the code. Ran means executed on a synthesized input, not that the code is correct.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 118 · with a code link: 55 · where Syntology ran a sample: 9 (9 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (9 of 118 tagged: 9 with a run with no instrument failure, 0 where every run was a failure of Syntology's instrument)
Page 1 of 2: papers 1 to 100 of 118, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
AdUE: Improving uncertainty estimation head for LoRA adapters in LLMs 21 May 2025 · 0 repositories · arXiv:2505.15443
-
LiteLMGuard: Seamless and Lightweight On-Device Prompt Filtering for Safeguarding Small Language Models against Quantization-induced Risks and Vulnerabilities 8 May 2025 · 2 repositories · arXiv:2505.05619
-
Sigma: A dataset for text-to-code semantic parsing with statistical analysis 5 Apr 2025 · 1 repository · arXiv:2504.04301
-
Comparative Analysis of Efficient Adapter-Based Fine-Tuning of State-of-the-Art Transformer Models 14 Jan 2025 · 0 repositories · arXiv:2501.08271
-
Aligning Brain Activity with Advanced Transformer Models: Exploring the Role of Punctuation in Semantic Processing 10 Jan 2025 · 1 repository · arXiv:2501.06278
-
IntegrityAI at GenAI Detection Task 2: Detecting Machine-Generated Academic Essays in English and Arabic Using ELECTRA and Stylometry 7 Jan 2025 · 0 repositories · arXiv:2501.05476
-
ELECTRA and GPT-4o: Cost-Effective Partners for Sentiment Analysis 29 Dec 2024 · 1 repository · arXiv:2501.00062
-
What Differentiates Educational Literature? A Multimodal Fusion Approach of Transformers and Computational Linguistics 26 Nov 2024 · 0 repositories · arXiv:2411.17593
-
A Comparative Analysis of Transformer and LSTM Models for Detecting Suicidal Ideation on Reddit 23 Nov 2024 · 1 repository · arXiv:2411.15404
-
LA4SR: illuminating the dark proteome with generative AI 11 Nov 2024 · 0 repositories · arXiv:2411.06798
-
A Named Entity Recognition and Topic Modeling-based Solution for Locating and Better Assessment of Natural Disasters in Social Media 1 May 2024 · 0 repositories · arXiv:2405.00903
-
Do "English" Named Entity Recognizers Work Well on Global Englishes? 20 Apr 2024 · 2 repositories · arXiv:2404.13465
-
FastLogAD: Log Anomaly Detection with Mask-Guided Pseudo Anomaly Generation and Discrimination 12 Apr 2024 · 1 repository · arXiv:2404.08750
-
Leveraging pre-trained language models for code generation 29 Feb 2024 · 1 repository
-
Are ELECTRA's Sentence Embeddings Beyond Repair? The Case of Semantic Textual Similarity 20 Feb 2024 · 1 repository · arXiv:2402.13130
-
A Curious Case of Searching for the Correlation between Training Data and Adversarial Robustness of Transformer Textual Models 18 Feb 2024 · 1 repository · arXiv:2402.11469
-
LogELECTRA: Self-supervised Anomaly Detection for Unstructured Logs 16 Feb 2024 · 0 repositories · arXiv:2402.10397
-
Dissecting vocabulary biases datasets through statistical testing and automated data augmentation for artifact mitigation in Natural Language Inference 14 Dec 2023 · 1 repository · arXiv:2312.08747
-
FA Team at the NTCIR-17 UFO Task 31 Oct 2023 · 0 repositories · arXiv:2310.20322
-
Fast-ELECTRA for Efficient Pre-training 11 Oct 2023 · 0 repositories · arXiv:2310.07347
-
Structural Self-Supervised Objectives for Transformers 15 Sep 2023 · 1 repository · arXiv:2309.08272
-
Transformer Based Punctuation Restoration for Turkish 15 Sep 2023 · 1 repository
-
Optimizing Multi-Class Text Classification: A Diverse Stacking Ensemble Framework Utilizing Transformers 19 Aug 2023 · 0 repositories · arXiv:2308.11519
-
An Ensemble Approach to Question Classification: Integrating Electra Transformer, GloVe, and LSTM 13 Aug 2023 · 0 repositories · arXiv:2308.06828
-
Analysis of the Evolution of Advanced Transformer-Based Language Models: Experiments on Opinion Mining 7 Aug 2023 · 1 repository · arXiv:2308.03235
-
Identifying Misinformation on YouTube through Transcript Contextual Analysis with Transformer Models 22 Jul 2023 · 1 repository · arXiv:2307.12155
-
Vacaspati: A Diverse Corpus of Bangla Literature 11 Jul 2023 · 0 repositories · arXiv:2307.05083
-
Comparison of Pre-trained Language Models for Turkish Address Parsing 24 Jun 2023 · 0 repositories · arXiv:2306.13947
-
Named entity recognition in resumes 22 Jun 2023 · 0 repositories · arXiv:2306.13062
-
Coreference-aware Double-channel Attention Network for Multi-party Dialogue Reading Comprehension 15 May 2023 · 1 repository · arXiv:2305.08348
-
Pre-training Language Model as a Multi-perspective Course Learner 6 May 2023 · 0 repositories · arXiv:2305.03981
-
Fine Tuning with Abnormal Examples 26 Apr 2023 · 0 repositories · arXiv:2304.13783
-
Technical Report: Impact of Position Bias on Language Models in Token Classification 26 Apr 2023 · 2 repositories · arXiv:2304.13567
-
Evaluation of Social Biases in Recent Large Pre-Trained Models 13 Apr 2023 · 0 repositories · arXiv:2304.06861
-
MUX-PLMs: Data Multiplexing for High-throughput Language Models 24 Feb 2023 · 1 repository · arXiv:2302.12441
-
Automatic Emotion Modelling in Written Stories 21 Dec 2022 · 1 repository · arXiv:2212.11382
-
LegalRelectra: Mixed-domain Language Modeling for Long-range Legal Text Comprehension 16 Dec 2022 · 0 repositories · arXiv:2212.08204
-
Probing for targeted syntactic knowledge through grammatical error detection 28 Oct 2022 · 1 repository · arXiv:2210.16228
-
Beyond English-Centric Bitexts for Better Multilingual Language Representation Learning 26 Oct 2022 · 0 repositories · arXiv:2210.14867
-
Effective Pre-Training Objectives for Transformer-based Autoencoders 24 Oct 2022 · 0 repositories · arXiv:2210.13536
-
Discriminative Language Model as Semantic Consistency Scorer for Prompt-based Few-Shot Text Classification 23 Oct 2022 · 0 repositories · arXiv:2210.12763
-
Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysis 10 Oct 2022 · 1 repository · arXiv:2210.04714Syntology official: harvested, nothing ran · 0 ran · 1 unverified (of 1 harvested sample)
-
UU-Tax at SemEval-2022 Task 3: Improving the generalizability of language models for taxonomy classification through data augmentation 7 Oct 2022 · 1 repository · arXiv:2210.03378
-
Downstream Datasets Make Surprisingly Good Pretraining Corpora 28 Sep 2022 · 1 repository · arXiv:2209.14389
-
YATO: Yet Another deep learning based Text analysis Open toolkit 28 Sep 2022 · 1 repository · arXiv:2209.13877
-
Probing for Understanding of English Verb Classes and Alternations in Large Pre-trained Language Models 11 Sep 2022 · 0 repositories · arXiv:2209.04811
-
No means ‘No’; a non-im-proper modeling approach, with embedded speculative context 30 Aug 2022 · 0 repositories
-
Building the Intent Landscape of Real-World Conversational Corpora with Extractive Question-Answering Transformers 26 Aug 2022 · 0 repositories · arXiv:2208.12886
-
Evaluate Confidence Instead of Perplexity for Zero-shot Commonsense Reasoning 23 Aug 2022 · 0 repositories · arXiv:2208.11007
-
ELECTRA is a Zero-Shot Learner, Too 17 Jul 2022 · 1 repository · arXiv:2207.08141Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples)
-
SimLM: Pre-training with Representation Bottleneck for Dense Passage Retrieval 6 Jul 2022 · 1 repository · arXiv:2207.02578
-
Prompting ELECTRA: Few-Shot Learning with Discriminative Pre-Trained Models 30 May 2022 · 1 repository · arXiv:2205.15223
-
Prompt Tuning for Discriminative Pre-trained Language Models 23 May 2022 · 1 repository · arXiv:2205.11166Syntology official (archive's flag): 7 ran · 7 ran (of which 4 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 2 where Syntology's instrument failed) · 5 unverified (of 12 harvested samples)
-
Pre-training Transformer Models with Sentence-Level Objectives for Answer Sentence Selection 20 May 2022 · 0 repositories · arXiv:2205.10455
-
SEMI-FND: Stacked Ensemble Based Multimodal Inference For Faster Fake News Detection 17 May 2022 · 0 repositories · arXiv:2205.08159
-
METRO: Efficient Denoising Pretraining of Large Scale Autoencoding Language Models with Model Generated Signals 13 Apr 2022 · 0 repositories · arXiv:2204.06644
-
Autoencoding Language Model Based Ensemble Learning for Commonsense Validation and Explanation 7 Apr 2022 · 0 repositories · arXiv:2204.03324
-
Pretraining Text Encoders with Adversarial Mixture of Training Signal Generators 7 Apr 2022 · 1 repository · arXiv:2204.03243Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 2 where Syntology's instrument failed) · 4 unverified (of 8 harvested samples) · 2 pointer-only (licence)
-
ANNA: Enhanced Language Representation for Question Answering 28 Mar 2022 · 0 repositories · arXiv:2203.14507
-
Transformer based ensemble for emotion detection 22 Mar 2022 · 0 repositories · arXiv:2203.11899
-
Towards Generalized Models for Task-oriented Dialogue Modeling on Spoken Conversations 8 Mar 2022 · 0 repositories · arXiv:2203.04045
-
Pre-trained Token-replaced Detection Model as Few-shot Learner 7 Mar 2022 · 1 repository · arXiv:2203.03235
-
Electra: Conditional Generative Model based Predicate-Aware Query Approximation 28 Jan 2022 · 0 repositories · arXiv:2201.12420
-
Context-Aware Prompt: Customize A Unique Prompt For Each Input 16 Jan 2022 · 0 repositories
-
Fine-Tuning Large Neural Language Models for Biomedical Natural Language Processing 15 Dec 2021 · 0 repositories · arXiv:2112.07869
-
A Comparative Study of Transformers on Word Sense Disambiguation 30 Nov 2021 · 0 repositories · arXiv:2111.15417
-
DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing 18 Nov 2021 · 3 repositories · arXiv:2111.09543Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 6 with no instrument failure: 0 honoured, 0 violated, 6 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples)
-
ANNA: Enhanced Language Representation for Question Answering 16 Nov 2021 · 0 repositories
-
Understanding Attention in Machine Reading Comprehension 16 Nov 2021 · 0 repositories
-
Maximizing Efficiency of Language Model Pre-training for Learning Representation 13 Oct 2021 · 0 repositories · arXiv:2110.06620
-
ASR Rescoring and Confidence Estimation with ELECTRA 5 Oct 2021 · 0 repositories · arXiv:2110.01857
-
Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids' Representations 13 Sep 2021 · 0 repositories · arXiv:2109.05958
-
FHAC at GermEval 2021: Identifying German toxic, engaging, and fact-claiming comments with ensemble learning 7 Sep 2021 · 1 repository · arXiv:2109.03094
-
Multilingual Multi-Aspect Explainability Analyses on Machine Reading Comprehension Models 26 Aug 2021 · 1 repository · arXiv:2108.11574
-
Relation Aware Semi-autoregressive Semantic Parsing for NL2SQL 2 Aug 2021 · 0 repositories · arXiv:2108.00804
-
IITK@LCP at SemEval-2021 Task 1: Classification for Lexical Complexity Regression Task 1 Aug 2021 · 0 repositories
-
PINGAN Omini-Sinitic at SemEval-2021 Task 4:Reading Comprehension of Abstract Meaning 1 Aug 2021 · 0 repositories
-
YoungSheldon at SemEval-2021 Task 5: Fine-tuning Pre-trained Language Models for Toxic Spans Detection using Token classification Objective 1 Aug 2021 · 1 repository
-
Using Knowledge-Embedded Attention to Augment Pre-trained Language Models for Fine-Grained Emotion Recognition 31 Jul 2021 · 1 repository · arXiv:2108.00194
-
Automatic Task Requirements Writing Evaluation via Machine Reading Comprehension 15 Jul 2021 · 1 repository · arXiv:2107.07957
-
An Initial Investigation of Non-Native Spoken Question-Answering 9 Jul 2021 · 0 repositories · arXiv:2107.04691
-
Learning to Sample Replacements for ELECTRA Pre-Training 25 Jun 2021 · 0 repositories · arXiv:2106.13715
-
SAS: Self-Augmentation Strategy for Language Model Pre-training 14 Jun 2021 · 1 repository · arXiv:2106.07176
-
BioELECTRA:Pretrained Biomedical text Encoder using Discriminators 11 Jun 2021 · 1 repository
-
SCRIPT: Self-Critic PreTraining of Transformers 1 Jun 2021 · 0 repositories
-
Training ELECTRA Augmented with Multi-word Selection 31 May 2021 · 0 repositories · arXiv:2106.00139
-
Discriminative Self-training for Punctuation Prediction 21 Apr 2021 · 0 repositories · arXiv:2104.10339
-
Efficient pre-training objectives for Transformers 20 Apr 2021 · 0 repositories · arXiv:2104.09694
-
WASSA@IITK at WASSA 2021: Multi-task Learning and Transformer Finetuning for Emotion Classification and Empathy Prediction 20 Apr 2021 · 0 repositories · arXiv:2104.09827
-
ELECTRAMed: a new pre-trained language representation model for biomedical NLP 19 Apr 2021 · 2 repositories · arXiv:2104.09585
-
Efficient transfer learning for NLP with ELECTRA 6 Apr 2021 · 1 repository · arXiv:2104.02756
-
MCL@IITK at SemEval-2021 Task 2: Multilingual and Cross-lingual Word-in-Context Disambiguation using Augmented Data, Signals, and Transformers 4 Apr 2021 · 0 repositories · arXiv:2104.01567
-
IITK@LCP at SemEval 2021 Task 1: Classification for Lexical Complexity Regression Task 2 Apr 2021 · 1 repository · arXiv:2104.01046
-
Let Your Heart Speak in its Mother Tongue: Multilingual Captioning of Cardiac Signals 19 Mar 2021 · 1 repository · arXiv:2103.11011
-
Robustly Optimized and Distilled Training for Natural Language Understanding 16 Mar 2021 · 0 repositories · arXiv:2103.08809
-
Analysis Of Contextual and Non-Contextual Word Embedding Models For Hindi NER With Web Application For Data Collection 18 Feb 2021 · 1 repository
-
COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining 16 Feb 2021 · 2 repositories · arXiv:2102.08473Syntology community repositories only · 5 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 4 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples)
-
Polyphone Disambiguation in Mandarin Chinese with Semi-Supervised Learning 1 Feb 2021 · 0 repositories · arXiv:2102.00621
-
Taking Notes on the Fly Helps Language Pre-Training 1 Jan 2021 · 0 repositories
-
A Multi-modal Deep Learning Model for Video Thumbnail Selection 31 Dec 2020 · 0 repositories · arXiv:2101.00073
Tasks archive 2025-07-28
20 shown of 136 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections