Methods › Natural Language Processing › Transformers › XLNet
XLNet
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
XLNet is an autoregressive Transformer that leverages the best of both autoregressive language modeling and autoencoding while attempting to avoid their limitations. Instead of using a fixed forward or backward factorization order as in conventional autoregressive models, XLNet maximizes the expected log likelihood of a sequence w.r.t. all possible permutations of the factorization order. Thanks to the permutation operation, the context for each position can consist of tokens from both left and right. In expectation, each position learns to utilize contextual information from all positions, i.e., capturing bidirectional context.
Additionally, inspired by the latest advancements in autogressive language modeling, XLNet integrates the segment recurrence mechanism and relative encoding scheme of Transformer-XL into pretraining, which empirically improves the performance especially for tasks involving a longer text sequence.
Papers archive 2025-07-28
30 shown of 167, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
Comparative sentiment analysis of public perception: Monkeypox vs. COVID-19 behavioral insights 12 May 2025 · 0 repositories · arXiv:2505.07430
-
A Character-based Diffusion Embedding Algorithm for Enhancing the Generation Quality of Generative Linguistic Steganographic Texts 2 May 2025 · 0 repositories · arXiv:2505.00977
-
Explainable AI for Sentiment Analysis of Human Metapneumovirus (HMPV) Using XLNet 1 Feb 2025 · 0 repositories · arXiv:2502.01663
-
Assessing Text Classification Methods for Cyberbullying Detection on Social Media Platforms 27 Dec 2024 · 0 repositories · arXiv:2412.19928
-
Feature Alignment-Based Knowledge Distillation for Efficient Compression of Large Language Models 27 Dec 2024 · 0 repositories · arXiv:2412.19449
-
Enhancing Multi-Class Disease Classification: Neoplasms, Cardiovascular, Nervous System, and Digestive Disorders Using Advanced LLMs 19 Nov 2024 · 0 repositories · arXiv:2411.12712
-
Rethinking Legal Judgement Prediction in a Realistic Scenario in the Era of Large Language Models 14 Oct 2024 · 1 repository · arXiv:2410.10542
-
What Matters in Explanations: Towards Explainable Fake Review Detection Focusing on Transformers 24 Jul 2024 · 0 repositories · arXiv:2407.21056
-
Why do you cite? An investigation on citation intents and decision-making classification processes 18 Jul 2024 · 0 repositories · arXiv:2407.13329
-
Ensemble Model With Bert,Roberta and Xlnet For Molecular property prediction 30 May 2024 · 0 repositories · arXiv:2406.06553
-
A Hybrid Deep Learning Framework for Stock Price Prediction Considering the Investor Sentiment of Online Forum Enhanced by Popularity 17 May 2024 · 0 repositories · arXiv:2405.10584
-
Eliciting Personality Traits in Large Language Models 13 Feb 2024 · 0 repositories · arXiv:2402.08341
-
Breaking Free Transformer Models: Task-specific Context Attribution Promises Improved Generalizability Without Fine-tuning Pre-trained LLMs 30 Jan 2024 · 1 repository · arXiv:2401.16638
-
An Assessment on Comprehending Mental Health through Large Language Models 9 Jan 2024 · 0 repositories · arXiv:2401.04592
-
Argumentation Element Annotation Modeling using XLNet 10 Nov 2023 · 0 repositories · arXiv:2311.06239
-
Modelling Sentiment Analysis: LLMs and data augmentation techniques 7 Nov 2023 · 1 repository · arXiv:2311.04139
-
An Ensemble Method Based on the Combination of Transformers with Convolutional Neural Networks to Detect Artificially Generated Text 26 Oct 2023 · 0 repositories · arXiv:2310.17312
-
Exploring Graph Neural Networks for Indian Legal Judgment Prediction 19 Oct 2023 · 0 repositories · arXiv:2310.12800
-
Lexical Squad@Multimodal Hate Speech Event Detection 2023: Multimodal Hate Speech Detection using Fused Ensemble Approach 23 Sep 2023 · 1 repository · arXiv:2309.13354
-
Simple is Better and Large is Not Enough: Towards Ensembling of Foundational Language Models 23 Aug 2023 · 0 repositories · arXiv:2308.12272
-
A Hybrid Machine Learning Model for Classifying Gene Mutations in Cancer using LSTM, BiLSTM, CNN, GRU, and GloVe 24 Jul 2023 · 0 repositories · arXiv:2307.14361
-
Resume Information Extraction via Post-OCR Text Processing 23 Jun 2023 · 0 repositories · arXiv:2306.13775
-
Research on Named Entity Recognition in Improved transformer with R-Drop structure 14 Jun 2023 · 0 repositories · arXiv:2306.08315
-
Domain-specific Continued Pretraining of Language Models for Capturing Long Context in Mental Health 20 Apr 2023 · 0 repositories · arXiv:2304.10447
-
Deep Learning for Opinion Mining and Topic Classification of Course Reviews 6 Apr 2023 · 0 repositories · arXiv:2304.03394
-
TrojText: Test-time Invisible Textual Trojan Insertion 3 Mar 2023 · 1 repository · arXiv:2303.02242Syntology ran 3 of 4 samples · 1 unverified · 4 pointer-only (licence)
-
HULAT at SemEval-2023 Task 10: Data augmentation for pre-trained transformers applied to the detection of sexism in social media 24 Feb 2023 · 1 repository · arXiv:2302.12840
-
A benchmark for toxic comment classification on Civil Comments dataset 26 Jan 2023 · 1 repository · arXiv:2301.11125
-
Analyzing Semantic Faithfulness of Language Models via Input Intervention on Question Answering 21 Dec 2022 · 1 repository · arXiv:2212.10696
-
Learning-To-Embed: Adopting Transformer based models for E-commerce Products Representation Learning 7 Dec 2022 · 0 repositories · arXiv:2212.03725
Tasks archive 2025-07-28
20 shown of 190 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections