{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2508-00924","title":"XAutoLM: Efficient Fine-Tuning of Language Models via Meta-Learning and AutoML","arxiv_id":"2508.00924","date":"2025-07-30","proceeding":"EMNLP","authors":["Ernesto L. Estevanell-Valladares","Suilan Estevez-Velarde","Yoan Gutiérrez","Andrés Montoyo","Ruslan Mitkov"],"abstract":"Experts in machine learning leverage domain knowledge to navigate decisions in model selection, hyperparameter optimization, and resource allocation. This is particularly critical for fine-tuning language models (LMs), where repeated trials incur substantial computational overhead and environmental impact. However, no existing automated framework simultaneously tackles the entire model selection and hyperparameter optimization (HPO) task for resource-efficient LM fine-tuning. We introduce XAutoLM, a meta-learning-augmented AutoML framework that reuses past experiences to optimize discriminative and generative LM fine-tuning pipelines efficiently. XAutoLM learns from stored successes and failures by extracting task- and system-level meta-features to bias its sampling toward valuable configurations and away from costly dead ends. On four text classification and two question-answering benchmarks, XAutoLM surpasses zero-shot optimizer's peak F1 on five of six tasks, cuts mean evaluation time of pipelines by up to 4.5x, reduces search error ratios by up to sevenfold, and uncovers up to 50% more pipelines above the zero-shot Pareto front. In contrast, simpler memory-based baselines suffer negative transfer. We release XAutoLM and our experience store to catalyze resource-efficient, Green AI fine-tuning in the NLP community.","url_abs":"https://arxiv.org/abs/2508.00924","url_pdf":"https://arxiv.org/pdf/2508.00924","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2508.00924","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2508.00924"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/EEstevanell/XAutoLM","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":4},"by_repo_kind":{"found_in_text":{"samples":4,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"05586bacf6cd1f03","entry":"compute_squad_f1","repo":"EEstevanell/XAutoLM","repo_kind":"found_in_text","path":"experiments/text_generation/src/execute_experiments.py","file_url":"https://github.com/EEstevanell/XAutoLM/blob/HEAD/experiments/text_generation/src/execute_experiments.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"05586bacf6cd1f03"}},{"code_sha256_prefix":"af2a06ad3fdaae35","entry":"is_categorical","repo":"EEstevanell/XAutoLM","repo_kind":"found_in_text","path":"autogoal-contrib/autogoal_contrib/autogoal_contrib/_utils.py","file_url":"https://github.com/EEstevanell/XAutoLM/blob/HEAD/autogoal-contrib/autogoal_contrib/autogoal_contrib/_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"af2a06ad3fdaae35"}},{"code_sha256_prefix":"e7563828076dfe8d","entry":"is_matrix_continuous_dense","repo":"EEstevanell/XAutoLM","repo_kind":"found_in_text","path":"autogoal-contrib/autogoal_contrib/autogoal_contrib/_utils.py","file_url":"https://github.com/EEstevanell/XAutoLM/blob/HEAD/autogoal-contrib/autogoal_contrib/autogoal_contrib/_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e7563828076dfe8d"}},{"code_sha256_prefix":"6061e8fb397f7d71","entry":"is_matrix_continuous_sparse","repo":"EEstevanell/XAutoLM","repo_kind":"found_in_text","path":"autogoal-contrib/autogoal_contrib/autogoal_contrib/_utils.py","file_url":"https://github.com/EEstevanell/XAutoLM/blob/HEAD/autogoal-contrib/autogoal_contrib/autogoal_contrib/_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6061e8fb397f7d71"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CL","source":"arxiv_api"},"syntology_extracted_results":null}