{"url":"/method/prelu","slug":"prelu","name":"PReLU","full_name":"Parameterized ReLU","full_name_withheld":false,"description_markdown":"A **Parametric Rectified Linear Unit**, or **PReLU**, is an activation function that generalizes the traditional rectified unit with a slope for negative values. Formally:\r\n\r\n$$f\\left(y\\_{i}\\right) = y\\_{i} \\text{ if } y\\_{i} \\ge 0$$\r\n$$f\\left(y\\_{i}\\right) = a\\_{i}y\\_{i} \\text{ if } y\\_{i} \\leq 0$$\r\n\r\nThe intuition is that different layers may require different types of nonlinearity. Indeed the authors find in experiments with convolutional neural networks that PReLus for the initial layer have more positive slopes, i.e. closer to linear. Since the filters of the first layers are Gabor-like filters such as edge or texture detectors, this shows a circumstance where positive and negative responses of filters are respected. In contrast the authors find deeper layers have smaller coefficients, suggesting the model becomes more discriminative at later layers (while it wants to retain more information at earlier layers).","description_state":"present","introduced_year":null,"introduced_by":{"title":"Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification","paper":"/paper/delving-deep-into-rectifiers-surpassing-human","first_author":"Kaiming He","n_authors":4,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/delving-deep-into-rectifiers-surpassing-human"},"source":{"url":"http://arxiv.org/abs/1502.01852v1","title":"Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/pytorch/pytorch/blob/96aaa311c0251d24decb9dc5da4957b7c590af6f/torch/nn/modules/activation.py#L968","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Activation Functions","url":"/methods/category/activation-functions","pwc_aliases":[]}],"n_papers_tagged":119,"archive_num_papers":119,"papers_newest_first":[{"paper":null,"title":"OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaning","date":"2025-05-31","arxiv_id":"2506.00338","n_code_links":0,"syntology":null},{"paper":null,"title":"Super-Resolution Generative Adversarial Networks based Video Enhancement","date":"2025-05-14","arxiv_id":"2505.10589","n_code_links":0,"syntology":null},{"paper":"/paper/ul-unas-ultra-lightweight-u-nets-for-real","title":"UL-UNAS: Ultra-Lightweight U-Nets for Real-Time Speech Enhancement via Network Architecture Search","date":"2025-03-01","arxiv_id":"2503.00340","n_code_links":1,"syntology":null},{"paper":"/paper/gompertz-linear-units-leveraging-asymmetry-1","title":"Gompertz Linear Units: Leveraging Asymmetry for Enhanced Learning Dynamics","date":"2025-02-05","arxiv_id":"2502.03654","n_code_links":1,"syntology":null},{"paper":"/paper/beyond-speaker-identity-text-guided-target","title":"Beyond Speaker Identity: Text Guided Target Speech Extraction","date":"2025-01-15","arxiv_id":"2501.09169","n_code_links":1,"syntology":null},{"paper":null,"title":"Uncertainty Estimation for Super-Resolution using ESRGAN","date":"2024-12-19","arxiv_id":"2412.15439","n_code_links":0,"syntology":null},{"paper":null,"title":"l0-Regularized Sparse Coding-based Interpretable Network for Multi-Modal Image Fusion","date":"2024-11-07","arxiv_id":"2411.04519","n_code_links":0,"syntology":null},{"paper":"/paper/deep-learning-based-ckm-construction-with","title":"Deep Learning-Based CKM Construction with Image Super-Resolution","date":"2024-10-28","arxiv_id":"2411.08887","n_code_links":1,"syntology":null},{"paper":"/paper/capsulenet-a-deep-learning-model-to-classify","title":"CapsuleNet: A Deep Learning Model To Classify GI Diseases Using EfficientNet-b7","date":"2024-10-24","arxiv_id":"2410.19151","n_code_links":1,"syntology":null},{"paper":"/paper/espnet-codec-comprehensive-training-and","title":"ESPnet-Codec: Comprehensive Training and Evaluation of Neural Codecs for Audio, Music, and Speech","date":"2024-09-24","arxiv_id":"2409.15897","n_code_links":2,"syntology":null},{"paper":"/paper/prelu-yet-another-single-layer-solution-to","title":"PReLU: Yet Another Single-Layer Solution to the XOR Problem","date":"2024-09-17","arxiv_id":"2409.10821","n_code_links":1,"syntology":null},{"paper":null,"title":"Two Stage Segmentation of Cervical Tumors using PocketNet","date":"2024-09-17","arxiv_id":"2409.11456","n_code_links":0,"syntology":null},{"paper":null,"title":"ESPnet-EZ: Python-only ESPnet for Easy Fine-tuning and Integration","date":"2024-09-14","arxiv_id":"2409.09506","n_code_links":0,"syntology":null},{"paper":null,"title":"LMAC-TD: Producing Time Domain Explanations for Audio Classifiers","date":"2024-09-13","arxiv_id":"2409.08655","n_code_links":0,"syntology":null},{"paper":null,"title":"The CHiME-8 DASR Challenge for Generalizable and Array Agnostic Distant Automatic Speech Recognition and Diarization","date":"2024-07-23","arxiv_id":"2407.16447","n_code_links":0,"syntology":null},{"paper":null,"title":"Early Explorations of Lightweight Models for Wound Segmentation on Mobile Devices","date":"2024-07-10","arxiv_id":"2407.07605","n_code_links":0,"syntology":null},{"paper":"/paper/noise-robust-speech-separation-with-fast","title":"Noise-robust Speech Separation with Fast Generative Correction","date":"2024-06-11","arxiv_id":"2406.07461","n_code_links":1,"syntology":null},{"paper":null,"title":"Deep Multi-Task Learning for Malware Image Classification","date":"2024-05-09","arxiv_id":"2405.05906","n_code_links":0,"syntology":null},{"paper":null,"title":"IFNet: Deep Imaging and Focusing for Handheld SAR with Millimeter-wave Signals","date":"2024-05-03","arxiv_id":"2405.02023","n_code_links":0,"syntology":null},{"paper":null,"title":"A cost minimization approach to fix the vocabulary size in a tokenizer for an End-to-End ASR system","date":"2024-04-29","arxiv_id":"2406.02563","n_code_links":0,"syntology":null},{"paper":null,"title":"Towards Efficient Resume Understanding: A Multi-Granularity Multi-Modal Pre-Training Approach","date":"2024-04-13","arxiv_id":"2404.13067","n_code_links":0,"syntology":null},{"paper":null,"title":"Power-Efficient Image Storage: Leveraging Super Resolution Generative Adversarial Network for Sustainable Compression and Reduced Carbon Footprint","date":"2024-04-06","arxiv_id":"2404.04642","n_code_links":0,"syntology":null},{"paper":null,"title":"Fully Data-Driven Model for Increasing Sampling Rate Frequency of Seismic Data using Super-Resolution Generative Adversarial Networks","date":"2024-01-31","arxiv_id":"2402.00153","n_code_links":0,"syntology":null},{"paper":"/paper/iterative-feedback-network-for-unsupervised","title":"Iterative Feedback Network for Unsupervised Point Cloud Registration","date":"2024-01-09","arxiv_id":"2401.04357","n_code_links":1,"syntology":null},{"paper":"/paper/docreal-robust-document-dewarping-of-real","title":"DocReal: Robust Document Dewarping of Real-Life Images via Attention-Enhanced Control Point Prediction","date":"2023-12-01","arxiv_id":null,"n_code_links":3,"syntology":null},{"paper":"/paper/target-oriented-domain-adaptation-for","title":"Texture and Noise Dual Adaptation for Infrared Image Super-Resolution","date":"2023-11-15","arxiv_id":"2311.08816","n_code_links":1,"syntology":null},{"paper":"/paper/resource-constrained-semantic-segmentation","title":"Resource Constrained Semantic Segmentation for Waste Sorting","date":"2023-10-30","arxiv_id":"2310.19407","n_code_links":1,"syntology":null},{"paper":null,"title":"Guided Frequency Loss for Image Restoration","date":"2023-09-27","arxiv_id":"2309.15563","n_code_links":0,"syntology":null},{"paper":null,"title":"A comparative analysis of SRGAN models","date":"2023-07-18","arxiv_id":"2307.09456","n_code_links":0,"syntology":null},{"paper":null,"title":"On Data Sampling Strategies for Training Neural Network Speech Separation Models","date":"2023-04-14","arxiv_id":"2304.07142","n_code_links":0,"syntology":null}],"papers_shown":30,"tasks":[{"task":"/task/super-resolution","name":"Super-Resolution","papers":31},{"task":"/task/image-super-resolution","name":"Image Super-Resolution","papers":21},{"task":"/task/semantic-segmentation","name":"Semantic Segmentation","papers":21},{"task":"/task/segmentation","name":"Segmentation","papers":14},{"task":null,"name":"Generative Adversarial Network","papers":12},{"task":"/task/speech-recognition","name":"Speech Recognition","papers":12},{"task":"/task/speech-separation","name":"Speech Separation","papers":12},{"task":"/task/speech-recognition-1","name":"speech-recognition","papers":11},{"task":"/task/automatic-speech-recognition-2","name":"Automatic Speech Recognition","papers":9},{"task":"/task/ssim","name":"SSIM","papers":9},{"task":"/task/automatic-speech-recognition","name":"Automatic Speech Recognition (ASR)","papers":8},{"task":"/task/decoder","name":"Decoder","papers":6},{"task":"/task/image-classification","name":"Image Classification","papers":6},{"task":"/task/image-classification","name":"image-classification","papers":6},{"task":"/task/autonomous-driving","name":"Autonomous Driving","papers":5},{"task":"/task/speech-enhancement","name":"Speech Enhancement","papers":5},{"task":"/task/object-detection","name":"Object Detection","papers":4},{"task":"/task/quantization","name":"Quantization","papers":4},{"task":"/task/real-time-semantic-segmentation","name":"Real-Time Semantic Segmentation","papers":4},{"task":"/task/object-detection-1","name":"object-detection","papers":4}],"tasks_shown":20,"n_tasks":116,"usage_by_year":[{"year":"2015","papers":1},{"year":"2016","papers":2},{"year":"2017","papers":3},{"year":"2018","papers":16},{"year":"2019","papers":11},{"year":"2020","papers":24},{"year":"2021","papers":13},{"year":"2022","papers":14},{"year":"2023","papers":11},{"year":"2024","papers":19},{"year":"2025","papers":5}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/prelu"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}