{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/statistical-estimation-of-malware-detection","title":"Statistical Estimation of Malware Detection Metrics in the Absence of Ground Truth","arxiv_id":"1810.07260","date":"2018-09-24","proceeding":null,"authors":["Du Pang","Sun Zheyuan","Chen Huashan","Cho Jin-Hee","Xu Shouhuai"],"abstract":"The accurate measurement of security metrics is a critical research problem\nbecause an improper or inaccurate measurement process can ruin the usefulness\nof the metrics, no matter how well they are defined. This is a highly\nchallenging problem particularly when the ground truth is unknown or noisy. In\ncontrast to the well perceived importance of defining security metrics, the\nmeasurement of security metrics has been little understood in the literature.\nIn this paper, we measure five malware detection metrics in the {\\em absence}\nof ground truth, which is a realistic setting that imposes many technical\nchallenges. The ultimate goal is to develop principled, automated methods for\nmeasuring these metrics at the maximum accuracy possible. The problem naturally\ncalls for investigations into statistical estimators by casting the measurement\nproblem as a {\\em statistical estimation} problem. We propose statistical\nestimators for these five malware detection metrics. By investigating the\nstatistical properties of these estimators, we are able to characterize when\nthe estimators are accurate, and what adjustments can be made to improve them\nunder what circumstances. We use synthetic data with known ground truth to\nvalidate these statistical estimators. Then, we employ these estimators to\nmeasure five metrics with respect to a large dataset collected from VirusTotal.\nWe believe our study touches upon a vital problem that has not been paid due\nattention and will inspire many future investigations.","url_abs":"http://arxiv.org/abs/1810.07260v1","url_pdf":"http://arxiv.org/pdf/1810.07260v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"statistical-estimation-of-malware-detection","repo_url":"https://github.com/Chenutsa/trustworthiness","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"malware-detection","task_name":"Malware Detection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}