{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-representations-of-ultrahigh","title":"Learning Representations of Ultrahigh-dimensional Data for Random Distance-based Outlier Detection","arxiv_id":"1806.04808","date":"2018-06-13","proceeding":null,"authors":["Guansong Pang","Longbing Cao","Ling Chen","Huan Liu"],"abstract":"Learning expressive low-dimensional representations of ultrahigh-dimensional\ndata, e.g., data with thousands/millions of features, has been a major way to\nenable learning methods to address the curse of dimensionality. However,\nexisting unsupervised representation learning methods mainly focus on\npreserving the data regularity information and learning the representations\nindependently of subsequent outlier detection methods, which can result in\nsuboptimal and unstable performance of detecting irregularities (i.e.,\noutliers).\n  This paper introduces a ranking model-based framework, called RAMODO, to\naddress this issue. RAMODO unifies representation learning and outlier\ndetection to learn low-dimensional representations that are tailored for a\nstate-of-the-art outlier detection approach - the random distance-based\napproach. This customized learning yields more optimal and stable\nrepresentations for the targeted outlier detectors. Additionally, RAMODO can\nleverage little labeled data as prior knowledge to learn more expressive and\napplication-relevant representations. We instantiate RAMODO to an efficient\nmethod called REPEN to demonstrate the performance of RAMODO.\n  Extensive empirical results on eight real-world ultrahigh dimensional data\nsets show that REPEN (i) enables a random distance-based detector to obtain\nsignificantly better AUC performance and two orders of magnitude speedup; (ii)\nperforms substantially better and more stably than four state-of-the-art\nrepresentation learning methods; and (iii) leverages less than 1% labeled data\nto achieve up to 32% AUC improvement.","url_abs":"http://arxiv.org/abs/1806.04808v1","url_pdf":"http://arxiv.org/pdf/1806.04808v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-representations-of-ultrahigh","repo_url":"https://github.com/GuansongPang/deep-outlier-detection","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":null},{"paper_slug":"learning-representations-of-ultrahigh","repo_url":"https://github.com/Ryosaeba8/Anomaly_detection","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"learning-representations-of-ultrahigh","repo_url":"https://github.com/xuhongzuo/DeepOD","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"anomaly-detection","task_name":"Anomaly Detection"},{"task_slug":"disease-prediction","task_name":"Disease Prediction"},{"task_slug":"network-intrusion-detection","task_name":"Network Intrusion Detection"},{"task_slug":"outlier-detection","task_name":"Outlier Detection"},{"task_slug":"representation-learning","task_name":"Representation Learning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1806.04808","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1806.04808"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xuhongzuo/DeepOD","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/GuansongPang/deep-outlier-detection","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Ryosaeba8/Anomaly_detection","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":3,"ran_fixture":1,"ran_honours":1},"by_repo_kind":{"listed":{"samples":5,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"d28c276a97503dfe","entry":"dataLoading","repo":"GuansongPang/deep-outlier-detection","repo_kind":"listed","path":"utilities.py","file_url":"https://github.com/GuansongPang/deep-outlier-detection/blob/HEAD/utilities.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d28c276a97503dfe"}},{"code_sha256_prefix":"69fde131ca536be7","entry":"get_priorknowledge_from_svmlight_file","repo":"GuansongPang/deep-outlier-detection","repo_kind":"listed","path":"rankod_sparse.py","file_url":"https://github.com/GuansongPang/deep-outlier-detection/blob/HEAD/rankod_sparse.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"69fde131ca536be7"}},{"code_sha256_prefix":"dae1d2f8eefb7bd2","entry":"lesinn","repo":"GuansongPang/deep-outlier-detection","repo_kind":"listed","path":"rankod.py","file_url":"https://github.com/GuansongPang/deep-outlier-detection/blob/HEAD/rankod.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dae1d2f8eefb7bd2"}},{"code_sha256_prefix":"a08b7b0032703e70","entry":"readMatdata","repo":"GuansongPang/deep-outlier-detection","repo_kind":"listed","path":"rankod_sparse.py","file_url":"https://github.com/GuansongPang/deep-outlier-detection/blob/HEAD/rankod_sparse.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a08b7b0032703e70"}},{"code_sha256_prefix":"925354d6b511cc5c","entry":"rescaling","repo":"GuansongPang/deep-outlier-detection","repo_kind":"listed","path":"utilities.py","file_url":"https://github.com/GuansongPang/deep-outlier-detection/blob/HEAD/utilities.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"925354d6b511cc5c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}