{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/benchmarking-automatic-machine-learning","title":"Benchmarking Automatic Machine Learning Frameworks","arxiv_id":"1808.06492","date":"2018-08-17","proceeding":null,"authors":["Adithya Balaji","Alexander Allen"],"abstract":"AutoML serves as the bridge between varying levels of expertise when\ndesigning machine learning systems and expedites the data science process. A\nwide range of techniques is taken to address this, however there does not exist\nan objective comparison of these techniques. We present a benchmark of current\nopen source AutoML solutions using open source datasets. We test auto-sklearn,\nTPOT, auto_ml, and H2O's AutoML solution against a compiled set of regression\nand classification datasets sourced from OpenML and find that auto-sklearn\nperforms the best across classification datasets and TPOT performs the best\nacross regression datasets.","url_abs":"http://arxiv.org/abs/1808.06492v1","url_pdf":"http://arxiv.org/pdf/1808.06492v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"benchmarking-automatic-machine-learning","repo_url":"https://github.com/ClimbsRocks/auto_ml","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":null},{"paper_slug":"benchmarking-automatic-machine-learning","repo_url":"https://github.com/EpistasisLab/tpot","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"LGPL-3.0"}},{"paper_slug":"benchmarking-automatic-machine-learning","repo_url":"https://github.com/automl/auto-sklearn","is_official":0,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"none","reach":{"status":"ok","spdx":"BSD-3-Clause"}},{"paper_slug":"benchmarking-automatic-machine-learning","repo_url":"https://github.com/h2oai/h2o-3","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"automl","task_name":"AutoML"},{"task_slug":"automated-feature-engineering","task_name":"Automated Feature Engineering"},{"task_slug":"machine-learning","task_name":"BIG-bench Machine Learning"},{"task_slug":"benchmarking","task_name":"Benchmarking"},{"task_slug":"classification-1","task_name":"Classification"},{"task_slug":"classification","task_name":"General Classification"},{"task_slug":"hyperparameter-optimization","task_name":"Hyperparameter Optimization"},{"task_slug":"regression-1","task_name":"regression"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":null,"mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}