{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/belief-a-distance-based-redundancy-proof","title":"BELIEF: A distance-based redundancy-proof feature selection method for Big Data","arxiv_id":"1804.05774","date":"2018-04-16","proceeding":null,"authors":["Sergio Ramírez-Gallego","Salvador García","Ning Xiong","Francisco Herrera"],"abstract":"With the advent of Big Data era, data reduction methods are highly demanded\ngiven its ability to simplify huge data, and ease complex learning processes.\nConcretely, algorithms that are able to filter relevant dimensions from a set\nof millions are of huge importance. Although effective, these techniques suffer\nfrom the \"scalability\" curse as well.\n  In this work, we propose a distributed feature weighting algorithm, which is\nable to rank millions of features in parallel using large samples. This method,\ninspired by the well-known RELIEF algorithm, introduces a novel redundancy\nelimination measure that provides similar schemes to those based on entropy at\na much lower cost. It also allows smooth scale up when more instances are\ndemanded in feature estimations. Empirical tests performed on our method show\nits estimation ability in manifold huge sets --both in number of features and\ninstances--, as well as its simplified runtime cost (specially, at the\nredundancy detection step).","url_abs":"http://arxiv.org/abs/1804.05774v1","url_pdf":"http://arxiv.org/pdf/1804.05774v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"belief-a-distance-based-redundancy-proof","repo_url":"https://github.com/sramirez/spark-RELIEFFC-fselection","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"feature-selection","task_name":"feature selection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}