{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/smu-smooth-activation-function-for-deep","title":"SMU: smooth activation function for deep networks using smoothing maximum technique","arxiv_id":"2111.04682","date":"2021-11-08","proceeding":null,"authors":["Koushik Biswas","Sandeep Kumar","Shilpak Banerjee","Ashish Kumar Pandey"],"abstract":"Deep learning researchers have a keen interest in proposing two new novel activation functions which can boost network performance. A good choice of activation function can have significant consequences in improving network performance. A handcrafted activation is the most common choice in neural network models. ReLU is the most common choice in the deep learning community due to its simplicity though ReLU has some serious drawbacks. In this paper, we have proposed a new novel activation function based on approximation of known activation functions like Leaky ReLU, and we call this function Smooth Maximum Unit (SMU). Replacing ReLU by SMU, we have got 6.22% improvement in the CIFAR100 dataset with the ShuffleNet V2 model.","url_abs":"https://arxiv.org/abs/2111.04682v2","url_pdf":"https://arxiv.org/pdf/2111.04682v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"smu-smooth-activation-function-for-deep","repo_url":"https://github.com/iFe1er/SMU","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"smu-smooth-activation-function-for-deep","repo_url":"https://github.com/iFe1er/SMU_pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"smu-smooth-activation-function-for-deep","repo_url":"https://github.com/lk18322280259/SMU","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"smu-smooth-activation-function-for-deep","repo_url":"https://github.com/MindCode-4/code-13/tree/main/smu","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"smu-smooth-activation-function-for-deep","repo_url":"https://github.com/pwc-1/Paper-9/tree/main/3/smu","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":null},{"paper_slug":"smu-smooth-activation-function-for-deep","repo_url":"https://github.com/reeered/SMU-MindSpore","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"mindspore","reach":{"status":"ok"}}],"tasks":[{"task_slug":"deep-learning","task_name":"Deep Learning"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"channel-shuffle","method_name":"Channel Shuffle"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"depthwise-convolution","method_name":"Depthwise Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"grouped-convolution","method_name":"Grouped Convolution"},{"method_slug":"groupwise-point-convolution","method_name":"Groupwise Point Convolution"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"pointwise-convolution","method_name":"Pointwise Convolution"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"shufflenet","method_name":"ShuffleNet"},{"method_slug":"shufflenet-block","method_name":"ShuffleNet Block"},{"method_slug":"shufflenet-v2-block","method_name":"ShuffleNet V2 Block"},{"method_slug":"shufflenet-v2-downsampling-block","method_name":"ShuffleNet V2 Downsampling Block"},{"method_slug":"shufflenet-v2","method_name":"ShuffleNet v2"},{"method_slug":"softmax","method_name":"Softmax"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2111.04682","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}