{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/feature-selection-with-optimal-coordinate","title":"Feature selection with optimal coordinate ascent (OCA)","arxiv_id":"1811.12064","date":"2018-11-29","proceeding":null,"authors":["David Saltiel","Eric Benhamou"],"abstract":"In machine learning, Feature Selection (FS) is a major part of efficient\nalgorithm. It fuels the algorithm and is the starting block for our prediction.\nIn this paper, we present a new method, called Optimal Coordinate Ascent (OCA)\nthat allows us selecting features among block and individual features. OCA\nrelies on coordinate ascent to find an optimal solution for gradient boosting\nmethods score (number of correctly classified samples). OCA takes into account\nthe notion of dependencies between variables forming blocks in our\noptimization. The coordinate ascent optimization solves the issue of the NP\nhard original problem where the number of combinations rapidly explode making a\ngrid search unfeasible. It reduces considerably the number of iterations\nchanging this NP hard problem into a polynomial search one. OCA brings\nsubstantial differences and improvements compared to previous coordinate ascent\nfeature selection method: we group variables into block and individual\nvariables instead of a binary selection. Our initial guess is based on the\nk-best group variables making our initial point more robust. We also introduced\nnew stopping criteria making our optimization faster. We compare these two\nmethods on our data set. We found that our method outperforms the initial one.\nWe also compare our method to the Recursive Feature Elimination (RFE) method\nand find that OCA leads to the minimum feature set with the highest score. This\nis a nice byproduct of our method as it provides empirically the most compact\ndata set with optimal performance.","url_abs":"http://arxiv.org/abs/1811.12064v3","url_pdf":"http://arxiv.org/pdf/1811.12064v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"feature-selection-with-optimal-coordinate","repo_url":"https://github.com/davidsaltiel/OCA","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":null}],"tasks":[{"task_slug":"feature-selection","task_name":"feature selection"}],"methods":[{"method_slug":"affine-coupling","method_name":"Affine Coupling"},{"method_slug":"normalizing-flows","method_name":"Normalizing Flows"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}