{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/llm-fe-automated-feature-engineering-for","title":"LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as Evolutionary Optimizers","arxiv_id":"2503.14434","date":"2025-03-18","proceeding":null,"authors":["Nikhil Abhyankar","Parshin Shojaee","Chandan K. Reddy"],"abstract":"Automated feature engineering plays a critical role in improving predictive model performance for tabular learning tasks. Traditional automated feature engineering methods are limited by their reliance on pre-defined transformations within fixed, manually designed search spaces, often neglecting domain knowledge. Recent advances using Large Language Models (LLMs) have enabled the integration of domain knowledge into the feature engineering process. However, existing LLM-based approaches use direct prompting or rely solely on validation scores for feature selection, failing to leverage insights from prior feature discovery experiments or establish meaningful reasoning between feature generation and data-driven performance. To address these challenges, we propose LLM-FE, a novel framework that combines evolutionary search with the domain knowledge and reasoning capabilities of LLMs to automatically discover effective features for tabular learning tasks. LLM-FE formulates feature engineering as a program search problem, where LLMs propose new feature transformation programs iteratively, and data-driven feedback guides the search process. Our results demonstrate that LLM-FE consistently outperforms state-of-the-art baselines, significantly enhancing the performance of tabular prediction models across diverse classification and regression benchmarks.","url_abs":"https://arxiv.org/abs/2503.14434v1","url_pdf":"https://arxiv.org/pdf/2503.14434v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"llm-fe-automated-feature-engineering-for","repo_url":"https://github.com/nikhilsab/llmfe","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"automated-feature-engineering","task_name":"Automated Feature Engineering"},{"task_slug":"feature-engineering","task_name":"Feature Engineering"},{"task_slug":"feature-selection","task_name":"feature selection"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2503.14434","atlas_url":"https://app.syntology.ai/?focus=2503.14434","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.14434"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/nikhilsab/llmfe","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":8,"unverified":3},"by_repo_kind":{"official":{"samples":11,"ran":8,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"e120137f6f7e0ddd","entry":"add_numba_decorator","repo":"nikhilsab/llmfe","repo_kind":"official","path":"llmfe/evaluator_accelerate.py","file_url":"https://github.com/nikhilsab/llmfe/blob/HEAD/llmfe/evaluator_accelerate.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e120137f6f7e0ddd"}},{"code_sha256_prefix":"a8d16181a84eee76","entry":"fill_missing","repo":"nikhilsab/llmfe","repo_kind":"official","path":"optimization_utils.py","file_url":"https://github.com/nikhilsab/llmfe/blob/HEAD/optimization_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a8d16181a84eee76"}},{"code_sha256_prefix":"d471862319b6d24a","entry":"is_categorical","repo":"nikhilsab/llmfe","repo_kind":"official","path":"optimization_utils.py","file_url":"https://github.com/nikhilsab/llmfe/blob/HEAD/optimization_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d471862319b6d24a"}},{"code_sha256_prefix":"13487f38a3a33dfd","entry":"preprocess_dataset","repo":"nikhilsab/llmfe","repo_kind":"official","path":"preprocessing.py","file_url":"https://github.com/nikhilsab/llmfe/blob/HEAD/preprocessing.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"13487f38a3a33dfd"}},{"code_sha256_prefix":"026999cb70a7da74","entry":"rename_function_calls","repo":"nikhilsab/llmfe","repo_kind":"official","path":"llmfe/code_manipulation.py","file_url":"https://github.com/nikhilsab/llmfe/blob/HEAD/llmfe/code_manipulation.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"026999cb70a7da74"}},{"code_sha256_prefix":"4d9bef27ef967fbe","entry":"serialize","repo":"nikhilsab/llmfe","repo_kind":"official","path":"utils.py","file_url":"https://github.com/nikhilsab/llmfe/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4d9bef27ef967fbe"}},{"code_sha256_prefix":"3cd7590893b13bbd","entry":"text_to_function","repo":"nikhilsab/llmfe","repo_kind":"official","path":"llmfe/code_manipulation.py","file_url":"https://github.com/nikhilsab/llmfe/blob/HEAD/llmfe/code_manipulation.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3cd7590893b13bbd"}},{"code_sha256_prefix":"32fba071d9cd1dc9","entry":"text_to_program","repo":"nikhilsab/llmfe","repo_kind":"official","path":"llmfe/code_manipulation.py","file_url":"https://github.com/nikhilsab/llmfe/blob/HEAD/llmfe/code_manipulation.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"32fba071d9cd1dc9"}},{"code_sha256_prefix":"932909d432058f1f","entry":"add_data_features","repo":"nikhilsab/llmfe","repo_kind":"official","path":"optimization_utils.py","file_url":"https://github.com/nikhilsab/llmfe/blob/HEAD/optimization_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"932909d432058f1f"}},{"code_sha256_prefix":"426af42ac029f32a","entry":"convert_categorical_to_integer_f","repo":"nikhilsab/llmfe","repo_kind":"official","path":"preprocessing.py","file_url":"https://github.com/nikhilsab/llmfe/blob/HEAD/preprocessing.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"426af42ac029f32a"}},{"code_sha256_prefix":"2d558b5afaec447b","entry":"create_mappings","repo":"nikhilsab/llmfe","repo_kind":"official","path":"preprocessing.py","file_url":"https://github.com/nikhilsab/llmfe/blob/HEAD/preprocessing.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2d558b5afaec447b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}