{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-concise-representations-for","title":"Learning concise representations for regression by evolving networks of trees","arxiv_id":"1807.00981","date":"2018-07-03","proceeding":"ICLR 2019 5","authors":["William La Cava","Tilak Raj Singh","James Taggart","Srinivas Suri","Jason H. Moore"],"abstract":"We propose and study a method for learning interpretable representations for\nthe task of regression. Features are represented as networks of multi-type\nexpression trees comprised of activation functions common in neural networks in\naddition to other elementary functions. Differentiable features are trained via\ngradient descent, and the performance of features in a linear model is used to\nweight the rate of change among subcomponents of each representation. The\nsearch process maintains an archive of representations with accuracy-complexity\ntrade-offs to assist in generalization and interpretation. We compare several\nstochastic optimization approaches within this framework. We benchmark these\nvariants on 100 open-source regression problems in comparison to\nstate-of-the-art machine learning approaches. Our main finding is that this\napproach produces the highest average test scores across problems while\nproducing representations that are orders of magnitude smaller than the next\nbest performing method (gradient boosting). We also report a negative result in\nwhich attempts to directly optimize the disentanglement of the representation\nresult in more highly correlated features.","url_abs":"http://arxiv.org/abs/1807.00981v3","url_pdf":"http://arxiv.org/pdf/1807.00981v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-concise-representations-for","repo_url":"https://github.com/lacava/feat","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"GPL-3.0"}},{"paper_slug":"learning-concise-representations-for","repo_url":"https://github.com/by1tTZ4IsQkAO80F/iclr_2019","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"learning-concise-representations-for","repo_url":"https://github.com/lacava/iclr_2019","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"learning-concise-representations-for","repo_url":"https://github.com/cavalab/feat","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"GPL-3.0"}},{"paper_slug":"learning-concise-representations-for","repo_url":"https://github.com/galdeia/feat","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"GPL-3.0"}}],"tasks":[{"task_slug":"disentanglement","task_name":"Disentanglement"},{"task_slug":"stochastic-optimization","task_name":"Stochastic Optimization"},{"task_slug":"regression-1","task_name":"regression"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1807.00981","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}