Methods › General › Regularization › Manifold Mixup
Manifold Mixup
Introduced by Vikas Verma et al. in Manifold Mixup: Better Representations by Interpolating Hidden States
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
Manifold Mixup is a regularization method that encourages neural networks to predict less confidently on interpolations of hidden representations. It leverages semantic interpolations as an additional training signal, obtaining neural networks with smoother decision boundaries at multiple levels of representation. As a result, neural networks trained with Manifold Mixup learn class-representations with fewer directions of variance.
Consider training a deep neural network f(x) = fₖ(gₖ(x)), where gₖ denotes the part of the neural network mapping the input data to the hidden representation at layer k, and fₖ denotes the part mapping such hidden representation to the output f(x). Training f using Manifold Mixup is performed in five steps:
(1) Select a random layer k from a set of eligible layers S in the neural network. This set may include the input layer g₀(x).
(2) Process two random data minibatches (x, y) and (x′, y′) as usual, until reaching layer k. This provides us with two intermediate minibatches (gₖ(x), y) and (gₖ(x′), y′).
(3) Perform Input Mixup on these intermediate minibatches. This produces the mixed minibatch:
(g̃ₖ, ỹ) = (Mix_λ(gₖ(x), gₖ(x′)), Mix_λ(y, y′)),
where Mix_λ(a, b) = λ·a + (1 − λ) ·b. Here, (y, y′ ) are one-hot labels, and the mixing coefficient λ∼Beta(α, α) as in mixup. For instance, α= 1.0 is equivalent to sampling λ∼U(0, 1).
(4) Continue the forward pass in the network from layer k until the output using the mixed minibatch (g̃ₖ, ỹ).
(5) This output is used to compute the loss value and gradients that update all the parameters of the neural network.
Papers archive 2025-07-28
26 shown of 26, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.
-
MEDUSA: A Multimodal Deep Fusion Multi-Stage Training Framework for Speech Emotion Recognition in Naturalistic Conditions 11 Jun 2025 · 1 repository · arXiv:2506.09556
-
A few-shot Label Unlearning in Vertical Federated Learning 14 Oct 2024 · 0 repositories · arXiv:2410.10922
-
PreMix: Addressing Label Scarcity in Whole Slide Image Classification with Pre-trained Multiple Instance Learning Aggregators 2 Aug 2024 · 0 repositories · arXiv:2408.01162
-
SynerMix: Synergistic Mixup Solution for Enhanced Intra-Class Cohesion and Inter-Class Separability in Image Classification 21 Mar 2024 · 2 repositories · arXiv:2403.14137
-
Mixture of Mixups for Multi-label Classification of Rare Anuran Sounds 14 Mar 2024 · 1 repository · arXiv:2403.09598
-
Improved Automatic Diabetic Retinopathy Severity Classification Using Deep Multimodal Fusion of UWF-CFP and OCTA Images 3 Oct 2023 · 0 repositories · arXiv:2310.01912
-
ShuffleMix: Improving Representations via Channel-Wise Shuffle of Interpolated Hidden States 30 May 2023 · 1 repository · arXiv:2305.18684
-
On the Effectiveness of Hybrid Pooling in Mixup-Based Graph Learning for Language Processing 6 Oct 2022 · 1 repository · arXiv:2210.03123
-
Set-based Meta-Interpolation for Few-Task Meta-Learning 20 May 2022 · 0 repositories · arXiv:2205.09990
-
Enhancing Cross-lingual Transfer by Manifold Mixup 9 May 2022 · 1 repository · arXiv:2205.04182
-
Learning to Classify Open Intent via Soft Labeling and Manifold Mixup 16 Apr 2022 · 1 repository · arXiv:2204.07804
-
STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation 20 Mar 2022 · 1 repository · arXiv:2203.10426Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
-
Noisy Feature Mixup 5 Oct 2021 · 2 repositories · arXiv:2110.02180Syntology ran 0 of 3 samples · 3 unverified
-
Coded-InvNet for Resilient Prediction Serving Systems 11 Jun 2021 · 0 repositories · arXiv:2106.06445
-
Distance Metric-Based Learning with Interpolated Latent Features for Location Classification in Endoscopy Image and Video 15 Mar 2021 · 0 repositories · arXiv:2103.08504
-
Robust Pollen Imagery Classification with Generative Modeling and Mixup Training 25 Feb 2021 · 0 repositories · arXiv:2102.13143
-
Logit As Auxiliary Weak-supervision for More Reliable and Accurate Prediction 1 Jan 2021 · 0 repositories
-
Regularizing Recurrent Neural Networks via Sequence Mixup 27 Nov 2020 · 0 repositories · arXiv:2012.07527
-
PointMixup: Augmentation for Point Clouds 14 Aug 2020 · 1 repository · arXiv:2008.06374
-
Remix: Rebalanced Mixup 8 Jul 2020 · 0 repositories · arXiv:2007.03943
-
Cross-Lingual Disaster-related Multi-label Tweet Classification with Manifold Mixup 1 Jul 2020 · 1 repository
-
Systematic Evaluation of Backdoor Data Poisoning Attacks on Image Classifiers 24 Apr 2020 · 0 repositories · arXiv:2004.11514
-
Charting the Right Manifold: Manifold Mixup for Few-shot Learning 28 Jul 2019 · 8 repositories · arXiv:1907.12087
-
Manifold Mixup: Learning Better Representations by Interpolating Hidden States 1 May 2019 · 1 repository
-
Manifold Mixup improves text recognition with CTC loss 11 Mar 2019 · 1 repository · arXiv:1903.04246
-
Manifold Mixup: Better Representations by Interpolating Hidden States 13 Jun 2018 · 12 repositories · arXiv:1806.05236Syntology ran 2 of 11 samples · 9 unverified · 5 pointer-only (licence)
Tasks archive 2025-07-28
20 shown of 57 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections