{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/comparison-of-non-linear-activation-functions","title":"Comparison of non-linear activation functions for deep neural networks on MNIST classification task","arxiv_id":"1804.02763","date":"2018-04-08","proceeding":null,"authors":["Dabal Pedamonti"],"abstract":"Activation functions play a key role in neural networks so it becomes\nfundamental to understand their advantages and disadvantages in order to\nachieve better performances. This paper will first introduce common types of\nnon linear activation functions that are alternative to the well known sigmoid\nfunction and then evaluate their characteristics. Moreover deeper neural\nnetworks will be analysed because they positively influence the final\nperformances compared to shallower networks. They also strictly depend on the\nweight initialisation hence the effect of drawing weights from Gaussian and\nuniform distribution will be analysed making particular attention on how the\nnumber of incoming and outgoing connection to a node influence the whole\nnetwork.","url_abs":"http://arxiv.org/abs/1804.02763v1","url_pdf":"http://arxiv.org/pdf/1804.02763v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"comparison-of-non-linear-activation-functions","repo_url":"https://github.com/Xiaohui9607/RFF_pytorch","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"comparison-of-non-linear-activation-functions","repo_url":"https://github.com/ken-power/CVND-FacialKeypointDetection","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"classification","task_name":"General Classification"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}