{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/deep-pipeline-embeddings-for-automl","title":"Deep Pipeline Embeddings for AutoML","arxiv_id":"2305.14009","date":"2023-05-23","proceeding":null,"authors":["Sebastian Pineda Arango","Josif Grabocka"],"abstract":"Automated Machine Learning (AutoML) is a promising direction for democratizing AI by automatically deploying Machine Learning systems with minimal human expertise. The core technical challenge behind AutoML is optimizing the pipelines of Machine Learning systems (e.g. the choice of preprocessing, augmentations, models, optimizers, etc.). Existing Pipeline Optimization techniques fail to explore deep interactions between pipeline stages/components. As a remedy, this paper proposes a novel neural architecture that captures the deep interaction between the components of a Machine Learning pipeline. We propose embedding pipelines into a latent representation through a novel per-component encoder mechanism. To search for optimal pipelines, such pipeline embeddings are used within deep-kernel Gaussian Process surrogates inside a Bayesian Optimization setup. Furthermore, we meta-learn the parameters of the pipeline embedding network using existing evaluations of pipelines on diverse collections of related datasets (a.k.a. meta-datasets). Through extensive experiments on three large-scale meta-datasets, we demonstrate that pipeline embeddings yield state-of-the-art results in Pipeline Optimization.","url_abs":"https://arxiv.org/abs/2305.14009v2","url_pdf":"https://arxiv.org/pdf/2305.14009v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"deep-pipeline-embeddings-for-automl","repo_url":"https://github.com/releaunifreiburg/DeepPipe","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"automl","task_name":"AutoML"},{"task_slug":"automatic-machine-learning-model-selection","task_name":"Automatic Machine Learning Model Selection"},{"task_slug":"bayesian-optimization","task_name":"Bayesian Optimization"},{"task_slug":"gaussian-processes","task_name":"Gaussian Processes"},{"task_slug":"hyperparameter-optimization","task_name":"Hyperparameter Optimization"}],"methods":[{"method_slug":"gaussian-process","method_name":"Gaussian Process"},{"method_slug":"fail","method_name":"fail"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}