{"url":"/method/tanhexp","slug":"tanhexp","name":"TanhExp","full_name":"Tanh Exponential Activation Function","full_name_withheld":false,"description_markdown":"Lightweight or mobile neural networks used for real-time computer vision tasks contain fewer parameters than normal\r\nnetworks, which lead to a constrained performance. In this work, we proposed a novel activation function named Tanh Exponential\r\nActivation Function (TanhExp) which can improve the performance for these networks on image classification task significantly.\r\nThe definition of TanhExp is $f(x) = x tanh(e^x)$. We demonstrate the simplicity, efficiency, and robustness of TanhExp on various\r\ndatasets and network models and TanhExp outperforms its counterparts in both convergence speed and accuracy. Its behaviour\r\nalso remains stable even with noise added and dataset altered. We show that without increasing the size of the network, the\r\ncapacity of lightweight neural networks can be enhanced by TanhExp with only a few training epochs and no extra parameters\r\nadded.","description_state":"present","introduced_year":null,"introduced_by":{"title":"TanhExp: A Smooth Activation Function with High Convergence Speed for Lightweight Neural Networks","paper":"/paper/tanhexp-a-smooth-activation-function-with","first_author":"Xinyu Liu","n_authors":2,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/tanhexp-a-smooth-activation-function-with"},"source":{"url":"https://arxiv.org/abs/2003.09855v2","title":"TanhExp: A Smooth Activation Function with High Convergence Speed for Lightweight Neural Networks","url_on_a_paper_host":true},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"General","area_id":"general","collection":"Activation Functions","url":"/methods/category/activation-functions","pwc_aliases":[]}],"n_papers_tagged":5,"archive_num_papers":5,"papers_newest_first":[{"paper":"/paper/neural-density-distance-fields","title":"Neural Density-Distance Fields","date":"2022-07-29","arxiv_id":"2207.14455","n_code_links":1,"syntology":null},{"paper":null,"title":"TeLU: A New Activation Function for Deep Learning","date":"2021-01-01","arxiv_id":null,"n_code_links":0,"syntology":null},{"paper":"/paper/smooth-activations-and-reproducibility-in-1","title":"Smooth activations and reproducibility in deep networks","date":"2020-10-20","arxiv_id":"2010.09931","n_code_links":4,"syntology":{"ran":0,"of":3,"unverified":3,"pointer_only":0}},{"paper":null,"title":"TanhSoft -- a family of activation functions combining Tanh and Softplus","date":"2020-09-08","arxiv_id":"2009.03863","n_code_links":0,"syntology":null},{"paper":"/paper/tanhexp-a-smooth-activation-function-with","title":"TanhExp: A Smooth Activation Function with High Convergence Speed for Lightweight Neural Networks","date":"2020-03-22","arxiv_id":"2003.09855","n_code_links":0,"syntology":null}],"papers_shown":5,"tasks":[{"task":"/task/image-classification","name":"Image Classification","papers":2},{"task":"/task/deep-learning","name":"Deep Learning","papers":1},{"task":"/task/classification","name":"General Classification","papers":1},{"task":"/task/nerf","name":"NeRF","papers":1},{"task":"/task/novel-view-synthesis","name":"Novel View Synthesis","papers":1},{"task":"/task/visual-localization","name":"Visual Localization","papers":1},{"task":"/task/image-classification","name":"image-classification","papers":1}],"tasks_shown":7,"n_tasks":7,"usage_by_year":[{"year":"2020","papers":3},{"year":"2021","papers":1},{"year":"2022","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/tanhexp"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}