{"url":"/method/manifold-mixup","slug":"manifold-mixup","name":"Manifold Mixup","full_name":"Manifold Mixup","full_name_withheld":false,"description_markdown":"**Manifold Mixup** is a regularization method that encourages neural networks to predict less confidently on interpolations of hidden representations. It leverages semantic interpolations as an additional training signal, obtaining neural networks with smoother decision boundaries at multiple levels of representation. As a result, neural networks trained with Manifold Mixup learn class-representations with fewer directions of variance.\r\n\r\nConsider training a deep neural network $f\\left(x\\right) = f\\_{k}\\left(g\\_{k}\\left(x\\right)\\right)$, where $g\\_{k}$ denotes the part of the neural network mapping the input data to the hidden representation at layer $k$, and $f\\_{k}$ denotes the\r\npart mapping such hidden representation to the output $f\\left(x\\right)$. Training $f$ using Manifold Mixup is performed in five steps:\r\n\r\n(1) Select a random layer $k$ from a set of eligible layers $S$ in the neural network. This set may include the input layer $g\\_{0}\\left(x\\right)$.\r\n\r\n(2) Process two random data minibatches $\\left(x, y\\right)$ and $\\left(x', y'\\right)$ as usual, until reaching layer $k$. This provides us with two intermediate minibatches $\\left(g\\_{k}\\left(x\\right), y\\right)$ and $\\left(g\\_{k}\\left(x'\\right), y'\\right)$.\r\n\r\n(3) Perform Input [Mixup](https://paperswithcode.com/method/mixup) on these intermediate minibatches. This produces the mixed minibatch:\r\n\r\n$$\r\n\\left(\\tilde{g}\\_{k}, \\tilde{y}\\right) = \\left(\\text{Mix}\\_{\\lambda}\\left(g\\_{k}\\left(x\\right), g\\_{k}\\left(x'\\right)\\right), \\text{Mix}\\_{\\lambda}\\left(y, y'\\right\r\n)\\right),\r\n$$\r\n\r\nwhere $\\text{Mix}\\_{\\lambda}\\left(a, b\\right) = \\lambda \\cdot a + \\left(1 − \\lambda\\right) \\cdot b$. Here, $\\left(y, y'\r\n\\right)$ are one-hot labels, and the mixing coefficient\r\n$\\lambda \\sim \\text{Beta}\\left(\\alpha, \\alpha\\right)$ as in mixup. For instance, $\\alpha = 1.0$ is equivalent to sampling $\\lambda \\sim U\\left(0, 1\\right)$.\r\n\r\n(4) Continue the forward pass in the network from layer $k$ until the output using the mixed minibatch $\\left(\\tilde{g}\\_{k}, \\tilde{y}\\right)$.\r\n\r\n(5) This output is used to compute the loss value and\r\ngradients that update all the parameters of the neural network.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Manifold Mixup: Better Representations by Interpolating Hidden States","paper":"/paper/manifold-mixup-better-representations-by","first_author":"Vikas Verma","n_authors":8,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/manifold-mixup-better-representations-by"},"source":{"url":"https://arxiv.org/abs/1806.05236v7","title":"Manifold Mixup: Better Representations by Interpolating Hidden States","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/vikasverma1077/manifold_mixup/blob/118ec709808b79dd336b10f4cf7deeacf541dfc7/supervised/models/resnet.py#L98","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Regularization","url":"/methods/category/regularization","pwc_aliases":[]}],"n_papers_tagged":26,"archive_num_papers":26,"papers_newest_first":[{"paper":"/paper/medusa-a-multimodal-deep-fusion-multi-stage","title":"MEDUSA: A Multimodal Deep Fusion Multi-Stage Training Framework for Speech Emotion Recognition in Naturalistic Conditions","date":"2025-06-11","arxiv_id":"2506.09556","n_code_links":1,"syntology":null},{"paper":null,"title":"A few-shot Label Unlearning in Vertical Federated Learning","date":"2024-10-14","arxiv_id":"2410.10922","n_code_links":0,"syntology":null},{"paper":null,"title":"PreMix: Addressing Label Scarcity in Whole Slide Image Classification with Pre-trained Multiple Instance Learning Aggregators","date":"2024-08-02","arxiv_id":"2408.01162","n_code_links":0,"syntology":null},{"paper":"/paper/improving-image-classification-accuracy","title":"SynerMix: Synergistic Mixup Solution for Enhanced Intra-Class Cohesion and Inter-Class Separability in Image Classification","date":"2024-03-21","arxiv_id":"2403.14137","n_code_links":2,"syntology":null},{"paper":"/paper/mixture-of-mixups-for-multi-label","title":"Mixture of Mixups for Multi-label Classification of Rare Anuran Sounds","date":"2024-03-14","arxiv_id":"2403.09598","n_code_links":1,"syntology":null},{"paper":null,"title":"Improved Automatic Diabetic Retinopathy Severity Classification Using Deep Multimodal Fusion of UWF-CFP and OCTA Images","date":"2023-10-03","arxiv_id":"2310.01912","n_code_links":0,"syntology":null},{"paper":"/paper/shufflemix-improving-representations-via","title":"ShuffleMix: Improving Representations via Channel-Wise Shuffle of Interpolated Hidden States","date":"2023-05-30","arxiv_id":"2305.18684","n_code_links":1,"syntology":null},{"paper":"/paper/enhancing-mixup-based-graph-learning-for","title":"On the Effectiveness of Hybrid Pooling in Mixup-Based Graph Learning for Language Processing","date":"2022-10-06","arxiv_id":"2210.03123","n_code_links":1,"syntology":null},{"paper":null,"title":"Set-based Meta-Interpolation for Few-Task Meta-Learning","date":"2022-05-20","arxiv_id":"2205.09990","n_code_links":0,"syntology":null},{"paper":"/paper/enhancing-cross-lingual-transfer-by-manifold-1","title":"Enhancing Cross-lingual Transfer by Manifold Mixup","date":"2022-05-09","arxiv_id":"2205.04182","n_code_links":1,"syntology":null},{"paper":"/paper/learning-to-classify-open-intent-via-soft","title":"Learning to Classify Open Intent via Soft Labeling and Manifold Mixup","date":"2022-04-16","arxiv_id":"2204.07804","n_code_links":1,"syntology":null},{"paper":"/paper/stemm-self-learning-with-speech-text-manifold","title":"STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation","date":"2022-03-20","arxiv_id":"2203.10426","n_code_links":1,"syntology":{"ran":1,"of":1,"unverified":0,"pointer_only":1}},{"paper":"/paper/noisy-feature-mixup","title":"Noisy Feature Mixup","date":"2021-10-05","arxiv_id":"2110.02180","n_code_links":2,"syntology":{"ran":0,"of":3,"unverified":3,"pointer_only":0}},{"paper":null,"title":"Coded-InvNet for Resilient Prediction Serving Systems","date":"2021-06-11","arxiv_id":"2106.06445","n_code_links":0,"syntology":null},{"paper":null,"title":"Distance Metric-Based Learning with Interpolated Latent Features for Location Classification in Endoscopy Image and Video","date":"2021-03-15","arxiv_id":"2103.08504","n_code_links":0,"syntology":null},{"paper":null,"title":"Robust Pollen Imagery Classification with Generative Modeling and Mixup Training","date":"2021-02-25","arxiv_id":"2102.13143","n_code_links":0,"syntology":null},{"paper":null,"title":"Logit As Auxiliary Weak-supervision for More Reliable and Accurate Prediction","date":"2021-01-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":null,"title":"Regularizing Recurrent Neural Networks via Sequence Mixup","date":"2020-11-27","arxiv_id":"2012.07527","n_code_links":0,"syntology":null},{"paper":"/paper/pointmixup-augmentation-for-point-clouds","title":"PointMixup: Augmentation for Point Clouds","date":"2020-08-14","arxiv_id":"2008.06374","n_code_links":1,"syntology":null},{"paper":null,"title":"Remix: Rebalanced Mixup","date":"2020-07-08","arxiv_id":"2007.03943","n_code_links":0,"syntology":null},{"paper":"/paper/cross-lingual-disaster-related-multi-label","title":"Cross-Lingual Disaster-related Multi-label Tweet Classification with Manifold Mixup","date":"2020-07-01","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"title":"Systematic Evaluation of Backdoor Data Poisoning Attacks on Image Classifiers","date":"2020-04-24","arxiv_id":"2004.11514","n_code_links":0,"syntology":null},{"paper":"/paper/charting-the-right-manifold-manifold-mixup","title":"Charting the Right Manifold: Manifold Mixup for Few-shot Learning","date":"2019-07-28","arxiv_id":"1907.12087","n_code_links":8,"syntology":null},{"paper":"/paper/manifold-mixup-learning-better","title":"Manifold Mixup: Learning Better Representations by Interpolating Hidden States","date":"2019-05-01","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":"/paper/manifold-mixup-improves-text-recognition-with","title":"Manifold Mixup improves text recognition with CTC loss","date":"2019-03-11","arxiv_id":"1903.04246","n_code_links":1,"syntology":null},{"paper":"/paper/manifold-mixup-better-representations-by","title":"Manifold Mixup: Better Representations by Interpolating Hidden States","date":"2018-06-13","arxiv_id":"1806.05236","n_code_links":12,"syntology":{"ran":2,"of":11,"unverified":9,"pointer_only":5}}],"papers_shown":26,"tasks":[{"task":"/task/data-augmentation","name":"Data Augmentation","papers":7},{"task":"/task/image-classification","name":"Image Classification","papers":6},{"task":"/task/image-classification","name":"image-classification","papers":5},{"task":"/task/classification","name":"General Classification","papers":3},{"task":"/task/few-shot-learning","name":"Few-Shot Learning","papers":2},{"task":"/task/multi-label-classification-2","name":"MUlTI-LABEL-ClASSIFICATION","papers":2},{"task":"/task/meta-learning","name":"Meta-Learning","papers":2},{"task":"/task/multi-label-classification","name":"Multi-Label Classification","papers":2},{"task":"/task/representation-learning","name":"Representation Learning","papers":2},{"task":"/task/text-classification","name":"Text Classification","papers":2},{"task":"/task/text-classification-1","name":"text-classification","papers":2},{"task":"/task/3d-point-cloud-classification","name":"3D Point Cloud Classification","papers":1},{"task":"/task/3d-point-cloud-data-augmentation","name":"3D Point Cloud Data Augmentation","papers":1},{"task":"/task/active-learning","name":"Active Learning","papers":1},{"task":"/task/anomaly-detection","name":"Anomaly Detection","papers":1},{"task":"/task/benchmarking","name":"Benchmarking","papers":1},{"task":"/task/bilevel-optimization","name":"Bilevel Optimization","papers":1},{"task":"/task/classification-1","name":"Classification","papers":1},{"task":"/task/code-classification","name":"Code Classification","papers":1},{"task":"/task/computational-efficiency","name":"Computational Efficiency","papers":1}],"tasks_shown":20,"n_tasks":57,"usage_by_year":[{"year":"2018","papers":1},{"year":"2019","papers":3},{"year":"2020","papers":5},{"year":"2021","papers":5},{"year":"2022","papers":5},{"year":"2023","papers":2},{"year":"2024","papers":4},{"year":"2025","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/manifold-mixup"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}