Methods › General › Ensembling › MoE
Mixture of Experts
MoE
Introduced by Neel Kanwal et al. in Equipping Computational Pathology Systems with Artifact Processing Pipelines: A Showcase for Computation and Performance Trade-offs
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
The archive carries no description for this method.
In Syntology about another paper, arXiv:2505.24449: repository known, no verified implementation (declared_code; repositories: 1 listed of 1 in the graph, strongest link: official). About another paper: this answer is about arXiv:2505.24449, while the archive credits arXiv:2403.07743.
From the lookup's note on this name: The same name also belongs to “Sparsely-Gated Mixture-of-Experts layer (MoE)”, a method in the paper “Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer” (arXiv:1701.06538, 5,455 citations), which the lookup finds cited far more than the paper this answer is about (6 citations), so this answer may be about a different sense of the name. For arXiv:1701.06538: its page on this site · ask the lookup about it.
Code Syntology holds under other names, as the lookup lists them (code_nearby); not a complete list, and none of them is this method.
- Soft MoE · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='Soft MoE') - TIME-MOE · shares_name_tokens (shares name tokens with this name; a different method) · level V3 · retrieve with
get_reference_implementation(name='TIME-MOE')
Origin paper of this answer, arXiv:2505.24449 (its page on this site · its page on Syntology): Syntology's run record for its harvested code, 4 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 4 where Syntology's instrument failed) · 2 unverified (of 6 harvested samples) · 2 pointer-only (licence). The instrument figure counts failures of Syntology's instrument, not of the code. Ran means executed on a synthesized input, not that the code is correct.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
What each level means, quoted from Syntology's server card: V0 “imports”; V1 “executes against a synthesized fixture and returns a value of the recorded type”; V2 “two implementations GENERATED from the paper's signature and spec agree numerically (1e-6) on fixture inputs” “neither of the two ever read the authors' own code, so V2 measures how unambiguously the paper specifies the method, not fidelity to the authors' implementation (measured 2026-09-11, commit 75eaa77: cross_checked_by is a second LLM on 2,628 of 2,831 served samples)”; V3 “paper-derived property tests pass on both of those same two generations.” The card adds: “No rung on this ladder compares the served code against the authors' implementation, so treat every level as evidence about runnability and internal agreement, not about matching the paper's results.” A level quoted in two parts is one definition the card writes as two clauses joined by a double hyphen, each clause quoted on its own (how these levels are quoted).
Papers archive 2025-07-28
archive papers tagged: 366 · with a code link: 142 · where Syntology ran a sample: 64 (48 with a run with no instrument failure, 16 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (64 of 366 tagged: 48 with a run with no instrument failure, 16 where every run was a failure of Syntology's instrument)
Page 1 of 4: papers 1 to 100 of 366, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
Mixture of Experts in Large Language Models 15 Jul 2025 · 0 repositories · arXiv:2507.11181
-
MoFE-Time: Mixture of Frequency Domain Experts for Time-Series Forecasting Models 9 Jul 2025 · 1 repository · arXiv:2507.06502
-
Growing Transformers: Modular Composition and Layer-wise Expansion on a Frozen Substrate 8 Jul 2025 · 1 repository · arXiv:2507.07129
-
Speech Quality Assessment Model Based on Mixture of Experts: System-Level Performance Enhancement and Utterance-Level Challenge Analysis 8 Jul 2025 · 0 repositories · arXiv:2507.06116
-
Learning Robust Stereo Matching in the Wild with Selective Mixture-of-Experts 7 Jul 2025 · 1 repository · arXiv:2507.04631
-
Sub-MoE: Efficient Mixture-of-Expert LLMs Compression via Subspace Expert Merging 29 Jun 2025 · 0 repositories · arXiv:2506.23266
-
Latent Prototype Routing: Achieving Near-Perfect Load Balancing in Mixture-of-Experts 26 Jun 2025 · 1 repository · arXiv:2506.21328
-
SAFEx: Analyzing Vulnerabilities of MoE-Based LLMs via Stable Safety-critical Expert Identification 20 Jun 2025 · 0 repositories · arXiv:2506.17368
-
Less is More: Undertraining Experts Improves Model Upcycling 17 Jun 2025 · 0 repositories · arXiv:2506.14126
-
LoRA-Mixer: Coordinate Modular LoRA Experts Through Serial Attention Routing 17 Jun 2025 · 0 repositories · arXiv:2507.00029
-
Ring-lite: Scalable Reasoning via C3PO-Stabilized Reinforcement Learning for LLMs 17 Jun 2025 · 0 repositories · arXiv:2506.14731
-
Utility-Driven Speculative Decoding for Mixture-of-Experts 17 Jun 2025 · 0 repositories · arXiv:2506.20675
-
EAQuant: Enhancing Post-Training Quantization for MoE Models via Expert-Aware Optimization 16 Jun 2025 · 0 repositories · arXiv:2506.13329
-
Serving Large Language Models on Huawei CloudMatrix384 15 Jun 2025 · 0 repositories · arXiv:2506.12708
-
Ming-Omni: A Unified Multimodal Model for Perception and Generation 11 Jun 2025 · 1 repository · arXiv:2506.09344Syntology official (archive's flag): 12 ran · 12 ran (of which 0 constructed an object rather than computing a result; 11 with no instrument failure: 3 honoured, 0 violated, 8 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 14 harvested samples)
-
M2Restore: Mixture-of-Experts-based Mamba-CNN Fusion Framework for All-in-One Image Restoration 9 Jun 2025 · 0 repositories · arXiv:2506.07814
-
MoE-GPS: Guidlines for Prediction Strategy for Dynamic Expert Duplication in MoE Load Balancing 9 Jun 2025 · 0 repositories · arXiv:2506.07366
-
SMAR: Soft Modality-Aware Routing Strategy for MoE-based Multimodal Large Language Models Preserving Language Capabilities 6 Jun 2025 · 0 repositories · arXiv:2506.06406
-
FlashDMoE: Fast Distributed MoE in a Single Kernel 5 Jun 2025 · 2 repositories · arXiv:2506.04667Syntology official (archive's flag): 3 ran · 3 ran (of which 2 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 0 where Syntology's instrument failed) · 4 unverified (of 7 harvested samples)
-
Out-of-Distribution Graph Models Merging 4 Jun 2025 · 0 repositories · arXiv:2506.03674
-
Decoding Knowledge Attribution in Mixture-of-Experts: A Framework of Basic-Refinement Collaboration and Efficiency Analysis 30 May 2025 · 0 repositories · arXiv:2505.24593
-
Mastering Massive Multi-Task Reinforcement Learning via Mixture-of-Expert Decision Transformer 30 May 2025 · 1 repository · arXiv:2505.24378
-
Mixture-of-Experts for Personalized and Semantic-Aware Next Location Prediction 30 May 2025 · 0 repositories · arXiv:2505.24597
-
On the Expressive Power of Mixture-of-Experts for Structured Complex Tasks 30 May 2025 · 0 repositories · arXiv:2505.24205
-
Advancing Expert Specialization for Better MoE 28 May 2025 · 0 repositories · arXiv:2505.22323
-
EvoMoE: Expert Evolution in Mixture of Experts for Multimodal Large Language Models 28 May 2025 · 0 repositories · arXiv:2505.23830
-
HiDream-I1: A High-Efficient Image Generative Foundation Model with Sparse Diffusion Transformer 28 May 2025 · 2 repositories · arXiv:2505.22705Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 1 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples)
-
FLAME-MoE: A Transparent End-to-End Research Platform for Mixture-of-Experts Language Models 26 May 2025 · 1 repository · arXiv:2505.20225
-
MoESD: Unveil Speculative Decoding's Potential for Accelerating Sparse MoE 26 May 2025 · 0 repositories · arXiv:2505.19645
-
Mosaic: Data-Free Knowledge Distillation via Mixture-of-Experts for Heterogeneous Distributed Environments 26 May 2025 · 0 repositories · arXiv:2505.19699
-
I2MoE: Interpretable Multimodal Interaction-aware Mixture-of-Experts 25 May 2025 · 1 repository · arXiv:2505.19190Syntology official (archive's flag): 4 ran · 4 ran (of which 4 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified; every one of the 4 samples that ran constructed an object rather than computing a result (of 4 harvested samples) · 4 pointer-only (licence)
-
Guiding the Experts: Semantic Priors for Efficient and Focused MoE Routing 24 May 2025 · 1 repository · arXiv:2505.18586
-
On Minimax Estimation of Parameters in Softmax-Contaminated Mixture of Experts 24 May 2025 · 0 repositories · arXiv:2505.18455
-
EvidenceMoE: A Physics-Guided Mixture-of-Experts with Evidential Critics for Advancing Fluorescence Light Detection and Ranging in Scattering Media 23 May 2025 · 0 repositories · arXiv:2505.21532
-
DriveMoE: Mixture-of-Experts for Vision-Language-Action Model in End-to-End Autonomous Driving 22 May 2025 · 0 repositories · arXiv:2505.16278
-
JanusDNA: A Powerful Bi-directional Hybrid DNA Foundation Model 22 May 2025 · 1 repository · arXiv:2505.17257Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 5 harvested samples) · 2 pointer-only (licence)
-
TrimR: Verifier-based Training-Free Thinking Compression for Efficient Test-Time Scaling 22 May 2025 · 0 repositories · arXiv:2505.17155
-
Chinese Toxic Language Mitigation via Sentiment Polarity Consistent Rewrites 21 May 2025 · 0 repositories · arXiv:2505.15297
-
Efficient Data Driven Mixture-of-Expert Extraction from Trained Networks 21 May 2025 · 0 repositories · arXiv:2505.15414
-
Hunyuan-TurboS: Advancing Large Language Models through Mamba-Transformer Synergy and Adaptive Chain-of-Thought 21 May 2025 · 0 repositories · arXiv:2505.15431
-
Not All Models Suit Expert Offloading: On Local Routing Consistency of Mixture-of-Expert Models 21 May 2025 · 1 repository · arXiv:2505.16056
-
EfficientLLM: Efficiency in Large Language Models 20 May 2025 · 0 repositories · arXiv:2505.13840
-
Local Mixtures of Experts: Essentially Free Test-Time Training via Model Merging 20 May 2025 · 1 repository · arXiv:2505.14136
-
THOR-MoE: Hierarchical Task-Guided and Context-Responsive Routing for Neural Machine Translation 20 May 2025 · 0 repositories · arXiv:2505.14173
-
Model Selection for Gaussian-gated Gaussian Mixture of Experts Using Dendrograms of Mixing Measures 19 May 2025 · 0 repositories · arXiv:2505.13052
-
Occult: Optimizing Collaborative Communication across Experts for Accelerated Parallel MoE Training and Inference 19 May 2025 · 1 repository · arXiv:2505.13345
-
TransferTraj: A Vehicle Trajectory Learning Model for Region and Task Transferability 19 May 2025 · 0 repositories · arXiv:2505.12672
-
On DeepSeekMoE: Statistical Benefits of Shared Experts and Normalized Sigmoid Gating 16 May 2025 · 0 repositories · arXiv:2505.10860
-
MoE-CAP: Benchmarking Cost, Accuracy and Performance of Sparse Mixture-of-Experts Systems 16 May 2025 · 0 repositories · arXiv:2505.11415
-
MegaScale-MoE: Large-Scale Communication-Efficient Training of Mixture-of-Experts Models in Production 16 May 2025 · 0 repositories · arXiv:2505.11432
-
PT-MoE: An Efficient Finetuning Framework for Integrating Mixture-of-Experts into Prompt Tuning 14 May 2025 · 1 repository · arXiv:2505.09519
-
Qwen3 Technical Report 14 May 2025 · 4 repositories · arXiv:2505.09388
-
UMoE: Unifying Attention and FFN with Shared Experts 12 May 2025 · 0 repositories · arXiv:2505.07260
-
FloE: On-the-Fly MoE Inference on Memory-constrained GPU 9 May 2025 · 0 repositories · arXiv:2505.05950
-
MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design 9 May 2025 · 1 repository · arXiv:2505.05799
-
Pangu Ultra MoE: How to Train Your Big MoE on Ascend NPUs 7 May 2025 · 0 repositories · arXiv:2505.04519
-
Faster MoE LLM Inference for Extremely Large Models 6 May 2025 · 0 repositories · arXiv:2505.03531
-
Multimodal Deep Learning-Empowered Beam Prediction in Future THz ISAC Systems 5 May 2025 · 0 repositories · arXiv:2505.02381
-
MoxE: Mixture of xLSTM Experts with Entropy-Aware Routing for Efficient Language Modeling 1 May 2025 · 0 repositories · arXiv:2505.01459
-
Accelerating Mixture-of-Experts Training with Adaptive Expert Replication 28 Apr 2025 · 0 repositories · arXiv:2504.19925
-
BadMoE: Backdooring Mixture-of-Experts LLMs via Optimizing Routing Triggers and Infecting Dormant Experts 24 Apr 2025 · 0 repositories · arXiv:2504.18598
-
MoE Parallel Folding: Heterogeneous Parallelism Mappings for Efficient Large-Scale MoE Model Training with Megatron Core 21 Apr 2025 · 0 repositories · arXiv:2504.14960
-
D²MoE: Dual Routing and Dynamic Scheduling for Efficient On-Device MoE-based LLM Serving 17 Apr 2025 · 0 repositories · arXiv:2504.15299
-
Dense Backpropagation Improves Training for Sparse Mixture-of-Experts 16 Apr 2025 · 1 repository · arXiv:2504.12463Syntology official (archive's flag): 4 ran · 4 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 0 violated, 1 with no contract checked; 3 where Syntology's instrument failed) · 2 unverified (of 6 harvested samples) · 6 pointer-only (licence)
-
Unveiling Hidden Collaboration within Mixture-of-Experts in Large Language Models 16 Apr 2025 · 0 repositories · arXiv:2504.12359
-
The Sword of Damocles in ViTs: Computational Redundancy Amplifies Adversarial Transferability 15 Apr 2025 · 0 repositories · arXiv:2504.10804
-
MoE-Lens: Towards the Hardware Limit of High-Throughput MoE LLM Serving Under Resource Constraints 12 Apr 2025 · 0 repositories · arXiv:2504.09345
-
C3PO: Critical-Layer, Core-Expert, Collaborative Pathway Optimization for Test-Time Expert Re-Mixing 10 Apr 2025 · 1 repository · arXiv:2504.07964
-
Cluster-Driven Expert Pruning for Mixture-of-Experts Large Language Models 10 Apr 2025 · 1 repository · arXiv:2504.07807
-
Holistic Capability Preservation: Towards Compact Yet Comprehensive Reasoning Models 9 Apr 2025 · 0 repositories · arXiv:2504.07158
-
MoEDiff-SR: Mixture of Experts-Guided Diffusion Model for Region-Adaptive MRI Super-Resolution 9 Apr 2025 · 1 repository · arXiv:2504.07308
-
Finding Fantastic Experts in MoEs: A Unified Study for Expert Dropping Strategies and Observations 8 Apr 2025 · 0 repositories · arXiv:2504.05586
-
HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference 8 Apr 2025 · 1 repository · arXiv:2504.05897
-
HeterMoE: Efficient Training of Mixture-of-Experts Models on Heterogeneous GPUs 4 Apr 2025 · 0 repositories · arXiv:2504.03871
-
MegaScale-Infer: Serving Mixture-of-Experts at Scale with Disaggregated Expert Parallelism 3 Apr 2025 · 0 repositories · arXiv:2504.02263
-
MiLo: Efficient Quantized MoE Inference with Mixture of Low-Rank Compensators 3 Apr 2025 · 1 repository · arXiv:2504.02658
-
Advancing MoE Efficiency: A Collaboration-Constrained Routing (C2R) Strategy for Better Expert Parallelism Design 2 Apr 2025 · 0 repositories · arXiv:2504.01337
-
Detecting Financial Fraud with Hybrid Deep Learning: A Mix-of-Experts Approach to Sequential and Anomalous Patterns 1 Apr 2025 · 0 repositories · arXiv:2504.03750
-
Mixture of Routers 30 Mar 2025 · 0 repositories · arXiv:2503.23362
-
Beyond Standard MoE: Mixture of Latent Experts for Resource-Efficient Language Models 29 Mar 2025 · 0 repositories · arXiv:2503.23100
-
MoQa: Rethinking MoE Quantization with Multi-stage Data-model Distribution Awareness 27 Mar 2025 · 0 repositories · arXiv:2503.21135
-
A multi-scale lithium-ion battery capacity prediction using mixture of experts and patch-based MLP 26 Mar 2025 · 1 repository · arXiv:2504.03706
-
Optimal Scaling Laws for Efficiency Gains in a Theoretical Transformer-Augmented Sectional MoE Framework 26 Mar 2025 · 0 repositories · arXiv:2503.20750
-
ExpertRAG: Efficient RAG with Mixture of Experts -- Optimizing Context Retrieval for Adaptive LLM Responses 23 Mar 2025 · 0 repositories · arXiv:2504.08744
-
Expert Race: A Flexible Routing Strategy for Scaling Diffusion Transformer with Mixture of Experts 20 Mar 2025 · 0 repositories · arXiv:2503.16057
-
Mixture of Lookup Experts 20 Mar 2025 · 1 repository · arXiv:2503.15798Syntology official (archive's flag): 2 ran · 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 1 unverified (of 3 harvested samples)
-
DiffMoE: Dynamic Token Selection for Scalable Diffusion Transformers 18 Mar 2025 · 0 repositories · arXiv:2503.14487
-
Frac-Connections: Fractional Extension of Hyper-Connections 18 Mar 2025 · 0 repositories · arXiv:2503.14125
-
Reinforcement learning-based motion imitation for physiologically plausible musculoskeletal motor control 18 Mar 2025 · 1 repository · arXiv:2503.14637
-
Samoyeds: Accelerating MoE Models with Structured Sparsity Leveraging Sparse Tensor Cores 13 Mar 2025 · 1 repository · arXiv:2503.10725
-
Double-Stage Feature-Level Clustering-Based Mixture of Experts Framework 12 Mar 2025 · 0 repositories · arXiv:2503.09504
-
MoE-Gen: High-Throughput MoE Inference on a Single GPU with Module-Based Batching 12 Mar 2025 · 0 repositories · arXiv:2503.09716
-
Priority-Aware Preemptive Scheduling for Mixed-Priority Workloads in MoE Inference 12 Mar 2025 · 0 repositories · arXiv:2503.09304
-
Accelerating MoE Model Inference with Expert Sharding 11 Mar 2025 · 0 repositories · arXiv:2503.08467
-
MoE-Loco: Mixture of Experts for Multitask Locomotion 11 Mar 2025 · 0 repositories · arXiv:2503.08564
-
A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications 10 Mar 2025 · 1 repository · arXiv:2503.07137Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)
-
ResMoE: Space-efficient Compression of Mixture of Experts LLMs via Residual Restoration 10 Mar 2025 · 1 repository · arXiv:2503.06881Syntology official (archive's flag): 10 ran · 10 ran (of which 0 constructed an object rather than computing a result; 7 with no instrument failure: 0 honoured, 1 violated, 6 with no contract checked; 3 where Syntology's instrument failed) · 5 unverified (of 15 harvested samples) · 1 pointer-only (licence)
-
MoFE: Mixture of Frozen Experts Architecture 9 Mar 2025 · 0 repositories · arXiv:2503.06491
-
GenieBlue: Integrating both Linguistic and Multimodal Capabilities for Large Language Models on Mobile Devices 8 Mar 2025 · 0 repositories · arXiv:2503.06019
-
Capacity-Aware Inference: Mitigating the Straggler Effect in Mixture of Experts 7 Mar 2025 · 0 repositories · arXiv:2503.05066
Tasks archive 2025-07-28
20 shown of 254 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
| Task | Papers |
|---|---|
| Mixture-of-Experts | 312 |
| GPU | 38 |
| Language Modelling | 34 |
| Computational Efficiency | 30 |
| Language Modeling | 29 |
| Large Language Model | 16 |
| Quantization | 14 |
| CPU | 13 |
| Multi-Task Learning | 13 |
| Image Classification | 12 |
| parameter-efficient fine-tuning | 12 |
| Scheduling | 11 |
| MMLU | 10 |
| Question Answering | 10 |
| Decoder | 9 |
| Diversity | 9 |
| image-classification | 9 |
| Continual Learning | 7 |
| Math | 7 |
| Retrieval | 7 |
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections