{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/on-the-reliable-detection-of-concept-drift","title":"On the Reliable Detection of Concept Drift from Streaming Unlabeled Data","arxiv_id":"1704.00023","date":"2017-03-31","proceeding":null,"authors":["Tegjyot Singh Sethi","Mehmed Kantardzic"],"abstract":"Classifiers deployed in the real world operate in a dynamic environment,\nwhere the data distribution can change over time. These changes, referred to as\nconcept drift, can cause the predictive performance of the classifier to drop\nover time, thereby making it obsolete. To be of any real use, these classifiers\nneed to detect drifts and be able to adapt to them, over time. Detecting drifts\nhas traditionally been approached as a supervised task, with labeled data\nconstantly being used for validating the learned model. Although effective in\ndetecting drifts, these techniques are impractical, as labeling is a difficult,\ncostly and time consuming activity. On the other hand, unsupervised change\ndetection techniques are unreliable, as they produce a large number of false\nalarms. The inefficacy of the unsupervised techniques stems from the exclusion\nof the characteristics of the learned classifier, from the detection process.\nIn this paper, we propose the Margin Density Drift Detection (MD3) algorithm,\nwhich tracks the number of samples in the uncertainty region of a classifier,\nas a metric to detect drift. The MD3 algorithm is a distribution independent,\napplication independent, model independent, unsupervised and incremental\nalgorithm for reliably detecting drifts from data streams. Experimental\nevaluation on 6 drift induced datasets and 4 additional datasets from the\ncybersecurity domain demonstrates that the MD3 approach can reliably detect\ndrifts, with significantly fewer false alarms compared to unsupervised feature\nbased drift detectors. The reduced false alarms enables the signaling of drifts\nonly when they are most likely to affect classification performance. As such,\nthe MD3 approach leads to a detection scheme which is credible, label efficient\nand general in its applicability.","url_abs":"http://arxiv.org/abs/1704.00023v1","url_pdf":"http://arxiv.org/pdf/1704.00023v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"on-the-reliable-detection-of-concept-drift","repo_url":"https://github.com/sarauppal/ConceptDrift_project","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"on-the-reliable-detection-of-concept-drift","repo_url":"https://github.com/mitre/menelaus/blob/dev/src/menelaus/concept_drift/md3.py","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"change-detection","task_name":"Change Detection"},{"task_slug":"drift-detection","task_name":"Drift Detection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/1704.00023","atlas_url":"https://app.syntology.ai/?focus=1704.00023","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}