{"url":"/method/maxout","slug":"maxout","name":"Maxout","full_name":"Maxout","full_name_withheld":false,"description_markdown":"The **Maxout Unit** is a generalization of the [ReLU](https://paperswithcode.com/method/relu) and the [leaky ReLU](https://paperswithcode.com/method/leaky-relu) functions. It is a piecewise linear function that returns the maximum of the inputs, designed to be used in conjunction with [dropout](https://paperswithcode.com/method/dropout). Both ReLU and leaky ReLU are special cases of Maxout. \r\n\r\n$$f\\left(x\\right) = \\max\\left(w^{T}\\_{1}x + b\\_{1}, w^{T}\\_{2}x + b\\_{2}\\right)$$\r\n\r\nThe main drawback of Maxout is that it is computationally expensive as it doubles the number of parameters for each neuron.","description_state":"present","introduced_year":null,"introduced_by":{"title":"Maxout Networks","paper":"/paper/maxout-networks","first_author":"Ian J. Goodfellow","n_authors":5,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/maxout-networks"},"source":{"url":"http://arxiv.org/abs/1302.4389v4","title":"Maxout Networks","url_on_a_paper_host":true},"code_snippet_url":"https://gist.github.com/daskol/05439f018465c8fb42ae547b8cc8a77b","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Activation Functions","url":"/methods/category/activation-functions","pwc_aliases":[]}],"n_papers_tagged":50,"archive_num_papers":50,"papers_newest_first":[{"paper":null,"title":"Deep-ICE: The first globally optimal algorithm for empirical risk minimization of two-layer maxout and ReLU networks","date":"2025-05-09","arxiv_id":"2505.05740","n_code_links":0,"syntology":null},{"paper":null,"title":"Depth-Bounds for Neural Networks via the Braid Arrangement","date":"2025-02-13","arxiv_id":"2502.09324","n_code_links":0,"syntology":null},{"paper":null,"title":"Deep Maxout Network-based Feature Fusion and Political Tangent Search Optimizer enabled Transfer Learning for Thalassemia Detection","date":"2023-08-03","arxiv_id":"2308.02029","n_code_links":0,"syntology":null},{"paper":"/paper/query-structure-modeling-for-inductive","title":"Query Structure Modeling for Inductive Logical Reasoning Over Knowledge Graphs","date":"2023-05-23","arxiv_id":"2305.13585","n_code_links":1,"syntology":null},{"paper":null,"title":"Error bounds for maxout neural network approximations of model predictive control","date":"2023-04-18","arxiv_id":"2304.08779","n_code_links":0,"syntology":null},{"paper":"/paper/expected-gradients-of-maxout-networks-and","title":"Expected Gradients of Maxout Networks and Consequences to Parameter Initialization","date":"2023-01-17","arxiv_id":"2301.06956","n_code_links":1,"syntology":null},{"paper":null,"title":"Deep Maxout Network Gaussian Process","date":"2022-08-08","arxiv_id":"2208.04468","n_code_links":0,"syntology":null},{"paper":null,"title":"On the Number of Regions of Piecewise Linear Neural Networks","date":"2022-06-17","arxiv_id":"2206.08615","n_code_links":0,"syntology":null},{"paper":"/paper/learning-with-stochastic-orders","title":"Learning with Stochastic Orders","date":"2022-05-27","arxiv_id":"2205.13684","n_code_links":1,"syntology":null},{"paper":null,"title":"Smooth Maximum Unit: Smooth Activation Function for Deep Networks Using Smoothing Maximum Technique","date":"2022-01-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/learning-discriminative-shrinkage-deep","title":"Learning Discriminative Shrinkage Deep Networks for Image Deconvolution","date":"2021-11-27","arxiv_id":"2111.13876","n_code_links":1,"syntology":null},{"paper":null,"title":"Time-Frequency Localization Using Deep Convolutional Maxout Neural Network in Persian Speech Recognition","date":"2021-08-09","arxiv_id":"2108.03818","n_code_links":0,"syntology":null},{"paper":"/paper/on-the-expected-complexity-of-maxout-networks","title":"On the Expected Complexity of Maxout Networks","date":"2021-07-01","arxiv_id":"2107.00379","n_code_links":1,"syntology":{"ran":3,"of":4,"unverified":1,"pointer_only":0}},{"paper":null,"title":"Revisiting 2D Convolutional Neural Networks for Graph-based Applications","date":"2021-05-23","arxiv_id":"2105.11016","n_code_links":0,"syntology":null},{"paper":null,"title":"Maximum and Leaky Maximum Propagation","date":"2021-05-21","arxiv_id":"2105.10277","n_code_links":0,"syntology":null},{"paper":null,"title":"Performance Evaluation of Deep Convolutional Maxout Neural Network in Speech Recognition","date":"2021-05-04","arxiv_id":"2105.01399","n_code_links":0,"syntology":null},{"paper":null,"title":"Sharp bounds for the number of regions of maxout networks and vertices of Minkowski sums","date":"2021-04-16","arxiv_id":"2104.08135","n_code_links":0,"syntology":null},{"paper":null,"title":"A new semi-supervised self-training method for lung cancer prediction","date":"2020-12-17","arxiv_id":"2012.09472","n_code_links":0,"syntology":null},{"paper":"/paper/activate-or-not-learning-customized","title":"Activate or Not: Learning Customized Activation","date":"2020-09-10","arxiv_id":"2009.04759","n_code_links":5,"syntology":null},{"paper":null,"title":"Deep Neural-Kernel Machines","date":"2020-07-13","arxiv_id":"2007.06655","n_code_links":0,"syntology":null},{"paper":"/paper/cspnet-a-new-backbone-that-can-enhance","title":"CSPNet: A New Backbone that can Enhance Learning Capability of CNN","date":"2019-11-27","arxiv_id":"1911.11929","n_code_links":123,"syntology":null},{"paper":null,"title":"Best Practices for Convolutional Neural Networks Applied to Object Recognition in Images","date":"2019-10-29","arxiv_id":"1910.13029","n_code_links":0,"syntology":null},{"paper":null,"title":"Graph-Preserving Grid Layout: A Simple Graph Drawing Method for Graph Classification using CNNs","date":"2019-09-26","arxiv_id":"1909.12383","n_code_links":0,"syntology":null},{"paper":null,"title":"Towards Learning Affine-Invariant Representations via Data-Efficient CNNs","date":"2019-08-31","arxiv_id":"1909.00114","n_code_links":0,"syntology":null},{"paper":null,"title":"Conditional Computation for Continual Learning","date":"2019-06-16","arxiv_id":"1906.06635","n_code_links":0,"syntology":null},{"paper":"/paper/max-plus-operators-applied-to-filter","title":"Max-plus Operators Applied to Filter Selection and Model Pruning in Neural Networks","date":"2019-03-19","arxiv_id":"1903.08072","n_code_links":1,"syntology":null},{"paper":"/paper/paragraph-level-neural-question-generation","title":"Paragraph-level Neural Question Generation with Maxout Pointer and Gated Self-attention Networks","date":"2018-10-01","arxiv_id":null,"n_code_links":1,"syntology":null},{"paper":null,"title":"Competition vs. Concatenation in Skip Connections of Fully Convolutional Networks","date":"2018-07-20","arxiv_id":"1807.07803","n_code_links":0,"syntology":null},{"paper":null,"title":"A Tropical Approach to Neural Networks with Piecewise Linear Activations","date":"2018-05-22","arxiv_id":"1805.08749","n_code_links":0,"syntology":null},{"paper":null,"title":"Single Image Super-Resolution Using Lightweight CNN with Maxout Units","date":"2017-11-07","arxiv_id":"1711.02321","n_code_links":0,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/image-classification","name":"Image Classification","papers":9},{"task":"/task/classification","name":"General Classification","papers":8},{"task":"/task/speech-recognition","name":"Speech Recognition","papers":4},{"task":"/task/image-classification","name":"image-classification","papers":4},{"task":"/task/speech-recognition-1","name":"speech-recognition","papers":4},{"task":"/task/data-augmentation","name":"Data Augmentation","papers":3},{"task":"/task/object-detection","name":"Object Detection","papers":3},{"task":"/task/semantic-segmentation","name":"Semantic Segmentation","papers":3},{"task":"/task/face-verification","name":"Face Verification","papers":2},{"task":"/task/graph-classification","name":"Graph Classification","papers":2},{"task":"/task/image-generation","name":"Image Generation","papers":2},{"task":"/task/language-modeling","name":"Language Modeling","papers":2},{"task":"/task/language-modelling","name":"Language Modelling","papers":2},{"task":"/task/object-recognition","name":"Object Recognition","papers":2},{"task":"/task/object-detection-1","name":"object-detection","papers":2},{"task":"/task/attribute","name":"Attribute","papers":1},{"task":"/task/machine-learning","name":"BIG-bench Machine Learning","papers":1},{"task":"/task/bayesian-inference","name":"Bayesian Inference","papers":1},{"task":"/task/computational-efficiency","name":"Computational Efficiency","papers":1},{"task":"/task/computed-tomography-ct","name":"Computed Tomography (CT)","papers":1}],"tasks_shown":20,"n_tasks":52,"usage_by_year":[{"year":"2013","papers":5},{"year":"2014","papers":2},{"year":"2015","papers":9},{"year":"2016","papers":3},{"year":"2017","papers":2},{"year":"2018","papers":3},{"year":"2019","papers":6},{"year":"2020","papers":3},{"year":"2021","papers":7},{"year":"2022","papers":4},{"year":"2023","papers":4},{"year":"2025","papers":2}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/maxout"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}