{"url":"/method/adact","slug":"adact","name":"AdAct","full_name":"Adaptive Activation","full_name_withheld":false,"description_markdown":"ADACT: LEARNING TO OPTIMIZE ACTIVATION FUNCTION CHOICE THROUGH ADAPTIVE ACTIVATION MODULES\r\n\r\nThis paper presents an innovative approach to enhancing neural network performance through the development and implementation of an adaptive activation function, termed adaptive activation (AdAct). AdAct amalgamates various wellestablished and novel activation functions into a single, learnable framework, allowing dynamic adaptation to specific network layers’ needs. We explore the effectiveness of ReLU and its variants, including ELU, LReLU, PReLU, RReLU, and more recent functions like Swish and Mish, integrating them into the AdAct function. Employing ConvNet variants across FMNIST, CIFAR10, SVHN and FER datasets, our study empirically assesses each function’s contribution and demonstrates AdAct’s potential in optimizing neural networks, especially in selecting optimal activation functions for diverse tasks.","description_state":"present","introduced_year":null,"introduced_by":{"title":null,"paper":null,"first_author":null,"n_authors":0,"url_abs":null,"archive_paper_url":null},"source":{"url":null,"title":null,"url_on_a_paper_host":false},"code_snippet_url":null,"code_snippet_url_on_a_code_host":false,"categories":[{"area":"Computer Vision","area_id":"computer-vision","collection":"Image Models","url":"/methods/category/image-models","pwc_aliases":[]}],"n_papers_tagged":1,"archive_num_papers":1,"papers_newest_first":[{"paper":null,"title":"Large Transformers are Better EEG Learners","date":"2023-08-20","arxiv_id":"2308.11654","n_code_links":0,"syntology":null}],"papers_shown":1,"tasks":[{"task":"/task/activity-recognition","name":"Activity Recognition","papers":1},{"task":"/task/eeg-1","name":"EEG","papers":1},{"task":"/task/eeg-decoding","name":"Eeg Decoding","papers":1},{"task":"/task/eeg","name":"Electroencephalogram (EEG)","papers":1},{"task":"/task/human-activity-recognition","name":"Human Activity Recognition","papers":1},{"task":"/task/time-series-1","name":"Time Series","papers":1},{"task":"/task/transfer-learning","name":"Transfer Learning","papers":1}],"tasks_shown":7,"n_tasks":7,"usage_by_year":[{"year":"2023","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/adact"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}