{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/spsa-fsr-simultaneous-perturbation-stochastic","title":"SPSA-FSR: Simultaneous Perturbation Stochastic Approximation for Feature Selection and Ranking","arxiv_id":"1804.05589","date":"2018-04-16","proceeding":null,"authors":["Zeren D. Yenice","Niranjan Adhikari","Yong Kai Wong","Vural Aksakalli","Alev Taskin Gumus","Babak Abbasi"],"abstract":"This manuscript presents the following: (1) an improved version of the Binary\nSimultaneous Perturbation Stochastic Approximation (SPSA) Method for feature\nselection in machine learning (Aksakalli and Malekipirbazari, Pattern\nRecognition Letters, Vol. 75, 2016) based on non-monotone iteration gains\ncomputed via the Barzilai and Borwein (BB) method, (2) its adaptation for\nfeature ranking, and (3) comparison against popular methods on public benchmark\ndatasets. The improved method, which we call SPSA-FSR, dramatically reduces the\nnumber of iterations required for convergence without impacting solution\nquality. SPSA-FSR can be used for feature ranking and feature selection both\nfor classification and regression problems. After a review of the current\nstate-of-the-art, we discuss our improvements in detail and present three sets\nof computational experiments: (1) comparison of SPSA-FS as a (wrapper) feature\nselection method against sequential methods as well as genetic algorithms, (2)\ncomparison of SPSA-FS as a feature ranking method in a classification setting\nagainst random forest importance, chi-squared, and information main methods,\nand (3) comparison of SPSA-FS as a feature ranking method in a regression\nsetting against minimum redundancy maximum relevance (MRMR), RELIEF, and linear\ncorrelation methods. The number of features in the datasets we use range from a\nfew dozens to a few thousands. Our results indicate that SPSA-FS converges to a\ngood feature set in no more than 100 iterations and therefore it is quite fast\nfor a wrapper method. SPSA-FS also outperforms popular feature selection as\nwell as feature ranking methods in majority of test cases, sometimes by a large\nmargin, and it stands as a promising new feature selection and ranking method.","url_abs":"http://arxiv.org/abs/1804.05589v1","url_pdf":"http://arxiv.org/pdf/1804.05589v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"spsa-fsr-simultaneous-perturbation-stochastic","repo_url":"https://github.com/pat-s/paper_hyperspectral","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"spsa-fsr-simultaneous-perturbation-stochastic","repo_url":"https://github.com/vaksakalli/spFtSel_py","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"spsa-fsr-simultaneous-perturbation-stochastic","repo_url":"https://github.com/vaksakalli/spfsr","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"spsa-fsr-simultaneous-perturbation-stochastic","repo_url":"https://github.com/vaksakalli/spsaml_py","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null},{"paper_slug":"spsa-fsr-simultaneous-perturbation-stochastic","repo_url":"https://github.com/zerenyenice/spsa-fsr","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":null}],"tasks":[{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"feature-selection","task_name":"feature selection"},{"task_slug":"regression-1","task_name":"regression"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}