{"url":"/method/self-adjusting-smooth-l1-loss","slug":"self-adjusting-smooth-l1-loss","name":"Self-Adjusting Smooth L1 Loss","full_name":"Self-Adjusting Smooth L1 Loss","full_name_withheld":false,"description_markdown":"**Self-Adjusting Smooth L1 Loss** is a loss function used in object detection that was introduced with [RetinaMask](https://paperswithcode.com/method/retinamask). This is an improved version of Smooth L1.  For Smooth L1 loss we have:\r\n\r\n$$ f(x) = 0.5  \\frac{x^{2}}{\\beta} \\text{ if } |x| < \\beta $$\r\n$$ f(x) = |x| -0.5\\beta \\text{ otherwise } $$\r\n\r\nHere a point $\\beta$ splits the positive axis range into two parts: $L2$ loss is used for targets in range $[0, \\beta]$, and $L1$ loss is used beyond $\\beta$ to avoid over-penalizing  utliers. The overall function is smooth (continuous, together with its derivative). However, the choice of control point ($\\beta$) is heuristic and is usually done by hyper parameter search.\r\n\r\nInstead, with self-adjusting smooth L1 loss, inside the loss function the running mean and variance of the absolute loss are recorded. We use the running minibatch mean and variance with a momentum of $0.9$ to update these two parameters.","description_state":"present","introduced_year":null,"introduced_by":{"title":"RetinaMask: Learning to predict masks improves state-of-the-art single-shot detection for free","paper":"/paper/retinamask-learning-to-predict-masks-improves","first_author":"Cheng-Yang Fu","n_authors":3,"url_abs":null,"archive_paper_url":"https://paperswithcode.com/paper/retinamask-learning-to-predict-masks-improves"},"source":{"url":"http://arxiv.org/abs/1901.03353v1","title":"RetinaMask: Learning to predict masks improves state-of-the-art single-shot detection for free","url_on_a_paper_host":true},"code_snippet_url":"https://github.com/chengyangfu/retinamask/blob/ce1eac2bb9882797dcba2a9bc72f994bec04fbdf/maskrcnn_benchmark/layers/adjust_smooth_l1_loss.py#L7","code_snippet_url_on_a_code_host":true,"categories":[{"area":"General","area_id":"general","collection":"Loss Functions","url":"/methods/category/loss-functions","pwc_aliases":[]}],"n_papers_tagged":1,"archive_num_papers":1,"papers_newest_first":[{"paper":"/paper/retinamask-learning-to-predict-masks-improves","title":"RetinaMask: Learning to predict masks improves state-of-the-art single-shot detection for free","date":"2019-01-10","arxiv_id":"1901.03353","n_code_links":53,"syntology":null}],"papers_shown":1,"tasks":[{"task":"/task/object-detection","name":"Object Detection","papers":1}],"tasks_shown":1,"n_tasks":1,"usage_by_year":[{"year":"2019","papers":1}],"row_source":"methods_table","archive":{"source":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","archive_url":"https://paperswithcode.com/method/self-adjusting-smooth-l1-loss"},"syntology_read_at":"2026-09-24T18:15:14+00:00"}