{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/anomalybench-an-open-benchmark-for","title":"Exathlon: A Benchmark for Explainable Anomaly Detection over Time Series","arxiv_id":"2010.05073","date":"2020-10-10","proceeding":null,"authors":["Vincent Jacob","Fei Song","Arnaud Stiegler","Bijan Rad","Yanlei Diao","Nesime Tatbul"],"abstract":"Access to high-quality data repositories and benchmarks have been instrumental in advancing the state of the art in many experimental research domains. While advanced analytics tasks over time series data have been gaining lots of attention, lack of such community resources severely limits scientific progress. In this paper, we present Exathlon, the first comprehensive public benchmark for explainable anomaly detection over high-dimensional time series data. Exathlon has been systematically constructed based on real data traces from repeated executions of large-scale stream processing jobs on an Apache Spark cluster. Some of these executions were intentionally disturbed by introducing instances of six different types of anomalous events (e.g., misbehaving inputs, resource contention, process failures). For each of the anomaly instances, ground truth labels for the root cause interval as well as those for the extended effect interval are provided, supporting the development and evaluation of a wide range of anomaly detection (AD) and explanation discovery (ED) tasks. We demonstrate the practical utility of Exathlon's dataset, evaluation methodology, and end-to-end data science pipeline design through an experimental study with three state-of-the-art AD and ED techniques.","url_abs":"https://arxiv.org/abs/2010.05073v3","url_pdf":"https://arxiv.org/pdf/2010.05073v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"anomalybench-an-open-benchmark-for","repo_url":"https://github.com/exathlonbenchmark/exathlon","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"anomaly-detection","task_name":"Anomaly Detection"},{"task_slug":"time-series-1","task_name":"Time Series"},{"task_slug":"time-series","task_name":"Time Series Analysis"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2010.05073","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2010.05073"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/exathlonbenchmark/exathlon","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":11},"by_repo_kind":{"official":{"samples":11,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5d66e8a5d1e948c6","entry":"add_executors_avg","repo":"exathlonbenchmark/exathlon","repo_kind":"official","path":"src/features/spark_alteration.py","file_url":"https://github.com/exathlonbenchmark/exathlon/blob/HEAD/src/features/spark_alteration.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5d66e8a5d1e948c6"}},{"code_sha256_prefix":"750421a410bf1fd4","entry":"add_nodes_avg","repo":"exathlonbenchmark/exathlon","repo_kind":"official","path":"src/features/spark_alteration.py","file_url":"https://github.com/exathlonbenchmark/exathlon/blob/HEAD/src/features/spark_alteration.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"750421a410bf1fd4"}},{"code_sha256_prefix":"8df6ac2e8b5c43bc","entry":"apply_alteration_bundle","repo":"exathlonbenchmark/exathlon","repo_kind":"official","path":"src/features/alteration.py","file_url":"https://github.com/exathlonbenchmark/exathlon/blob/HEAD/src/features/alteration.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8df6ac2e8b5c43bc"}},{"code_sha256_prefix":"5049ce852eef782b","entry":"apply_identity","repo":"exathlonbenchmark/exathlon","repo_kind":"official","path":"src/features/alteration.py","file_url":"https://github.com/exathlonbenchmark/exathlon/blob/HEAD/src/features/alteration.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5049ce852eef782b"}},{"code_sha256_prefix":"19f79fb01d21ed69","entry":"get_altered_features","repo":"exathlonbenchmark/exathlon","repo_kind":"official","path":"src/features/alteration.py","file_url":"https://github.com/exathlonbenchmark/exathlon/blob/HEAD/src/features/alteration.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"19f79fb01d21ed69"}},{"code_sha256_prefix":"a0ecada557f84cff","entry":"get_auc","repo":"exathlonbenchmark/exathlon","repo_kind":"official","path":"src/metrics/ad_evaluators.py","file_url":"https://github.com/exathlonbenchmark/exathlon/blob/HEAD/src/metrics/ad_evaluators.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a0ecada557f84cff"}},{"code_sha256_prefix":"add5669535af5e9a","entry":"get_f_beta_score","repo":"exathlonbenchmark/exathlon","repo_kind":"official","path":"src/metrics/ad_evaluators.py","file_url":"https://github.com/exathlonbenchmark/exathlon/blob/HEAD/src/metrics/ad_evaluators.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"add5669535af5e9a"}},{"code_sha256_prefix":"7412988e38b4679e","entry":"get_range_pr_params","repo":"exathlonbenchmark/exathlon","repo_kind":"official","path":"src/metrics/ad_evaluators.py","file_url":"https://github.com/exathlonbenchmark/exathlon/blob/HEAD/src/metrics/ad_evaluators.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"7412988e38b4679e"}},{"code_sha256_prefix":"fdb76e77bc9f6072","entry":"get_scaler","repo":"exathlonbenchmark/exathlon","repo_kind":"official","path":"src/features/transformers.py","file_url":"https://github.com/exathlonbenchmark/exathlon/blob/HEAD/src/features/transformers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fdb76e77bc9f6072"}},{"code_sha256_prefix":"93a1ef6fb5dc27c0","entry":"threshold_period_scores","repo":"exathlonbenchmark/exathlon","repo_kind":"official","path":"src/detection/helpers.py","file_url":"https://github.com/exathlonbenchmark/exathlon/blob/HEAD/src/detection/helpers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"93a1ef6fb5dc27c0"}},{"code_sha256_prefix":"76feb0fb8daaa850","entry":"unravel_periods","repo":"exathlonbenchmark/exathlon","repo_kind":"official","path":"src/features/transformers.py","file_url":"https://github.com/exathlonbenchmark/exathlon/blob/HEAD/src/features/transformers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"76feb0fb8daaa850"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}