{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/loghub-a-large-collection-of-system-log","title":"Loghub: A Large Collection of System Log Datasets for AI-driven Log Analytics","arxiv_id":"2008.06448","date":"2020-08-14","proceeding":null,"authors":["Jieming Zhu","Shilin He","Pinjia He","Jinyang Liu","Michael R. Lyu"],"abstract":"Logs have been widely adopted in software system development and maintenance because of the rich runtime information they record. In recent years, the increase of software size and complexity leads to the rapid growth of the volume of logs. To handle these large volumes of logs efficiently and effectively, a line of research focuses on developing intelligent and automated log analysis techniques. However, only a few of these techniques have reached successful deployments in industry due to the lack of public log datasets and open benchmarking upon them. To fill this significant gap and facilitate more research on AI-driven log analytics, we have collected and released loghub, a large collection of system log datasets. In particular, loghub provides 19 real-world log datasets collected from a wide range of software systems, including distributed systems, supercomputers, operating systems, mobile systems, server applications, and standalone software. In this paper, we summarize the statistics of these datasets, introduce some practical usage scenarios of the loghub datasets, and present our benchmarking results on loghub to benefit the researchers and practitioners in this field. Up to the time of this paper writing, the loghub datasets have been downloaded for roughly 90,000 times in total by hundreds of organizations from both industry and academia. The loghub datasets are available at https://github.com/logpai/loghub.","url_abs":"https://arxiv.org/abs/2008.06448v3","url_pdf":"https://arxiv.org/pdf/2008.06448v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"links_only","authors_date_abstract":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license), from the Kaggle arXiv metadata snapshot of 2026-09-12"},"code_links":[{"paper_slug":"loghub-a-large-collection-of-system-log","repo_url":"https://github.com/logpai/loghub","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"loghub-a-large-collection-of-system-log","repo_url":"https://github.com/Netflix/derand","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"loghub-a-large-collection-of-system-log","repo_url":"https://github.com/R-SPIRIT/Domain-LOA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"loghub-a-large-collection-of-system-log","repo_url":"https://github.com/WraySmith/log-anomaly","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"loghub-a-large-collection-of-system-log","repo_url":"https://github.com/forallx94/log_anomaly_detection","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}},{"paper_slug":"loghub-a-large-collection-of-system-log","repo_url":"https://github.com/logpai/loghub-2.0","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"loghub-a-large-collection-of-system-log","repo_url":"https://github.com/logpai/logpub","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"NOASSERTION"}},{"paper_slug":"loghub-a-large-collection-of-system-log","repo_url":"https://github.com/spilabkorea/Domain-LOA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2008.06448","atlas_url":"https://app.syntology.ai/?focus=2008.06448","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2008.06448"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/R-SPIRIT/Domain-LOA","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Netflix/derand","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/logpai/loghub","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/WraySmith/log-anomaly","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/logpai/logpub","reach":{"status":"ok","spdx":"NOASSERTION"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/forallx94/log_anomaly_detection","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/spilabkorea/Domain-LOA","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/logpai/loghub-2.0","reach":null}],"summary":{"ran":6,"ran_draft_wrong":1},"by_repo_kind":{"listed":{"samples":7,"ran":7,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"c1d297b6d97af37d","entry":"collect_event_ids","repo":"WraySmith/log-anomaly","repo_kind":"listed","path":"process/sliding_window_processor.py","file_url":"https://github.com/WraySmith/log-anomaly/blob/HEAD/process/sliding_window_processor.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c1d297b6d97af37d"}},{"code_sha256_prefix":"081605aaa2a31f86","entry":"comExit","repo":"WraySmith/log-anomaly","repo_kind":"listed","path":"parse/logparser/LKE/LKE.py","file_url":"https://github.com/WraySmith/log-anomaly/blob/HEAD/parse/logparser/LKE/LKE.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"081605aaa2a31f86"}},{"code_sha256_prefix":"04e9124a19f95d56","entry":"get_accuracy","repo":"logpai/loghub-2.0","repo_kind":"listed","path":"benchmark/LogPPT/logppt/evaluation/evaluator.py","file_url":"https://github.com/logpai/loghub-2.0/blob/HEAD/benchmark/LogPPT/logppt/evaluation/evaluator.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"04e9124a19f95d56"}},{"code_sha256_prefix":"18ea37d0aa8f9bca","entry":"listContained","repo":"WraySmith/log-anomaly","repo_kind":"listed","path":"parse/logparser/LKE/LKE.py","file_url":"https://github.com/WraySmith/log-anomaly/blob/HEAD/parse/logparser/LKE/LKE.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"18ea37d0aa8f9bca"}},{"code_sha256_prefix":"1fb2cf516043ba9f","entry":"posiToSplit","repo":"WraySmith/log-anomaly","repo_kind":"listed","path":"parse/logparser/LKE/LKE.py","file_url":"https://github.com/WraySmith/log-anomaly/blob/HEAD/parse/logparser/LKE/LKE.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1fb2cf516043ba9f"}},{"code_sha256_prefix":"1d16cdb68d278df7","entry":"sequence_padder","repo":"WraySmith/log-anomaly","repo_kind":"listed","path":"process/sliding_window_processor.py","file_url":"https://github.com/WraySmith/log-anomaly/blob/HEAD/process/sliding_window_processor.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1d16cdb68d278df7"}},{"code_sha256_prefix":"535ebf130648bbc5","entry":"windower","repo":"WraySmith/log-anomaly","repo_kind":"listed","path":"process/sliding_window_processor.py","file_url":"https://github.com/WraySmith/log-anomaly/blob/HEAD/process/sliding_window_processor.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"535ebf130648bbc5"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}