{"url":"/task/hyperparameter-optimization","name":"Hyperparameter Optimization","slug":"hyperparameter-optimization","description_markdown":"**Hyperparameter Optimization** is the problem of choosing a set of optimal hyperparameters for a learning algorithm. Whether the algorithm is suitable for the data directly depends on hyperparameters, which directly influence overfitting or underfitting. Each model requires different assumptions, weights or training speeds for different types of data under the conditions of a given loss function.\r\n\r\n\r\n<span class=\"description-source\">Source: [Data-driven model for fracturing design optimization: focus on building digital database and production forecast ](https://arxiv.org/abs/1910.14499)</span>","categories":[{"name":"Methodology","url":"/area/methodology"}],"source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","slug_source":"archive_url"},"counts":{"papers_tagged":813,"papers_with_code":339,"benchmarks":1,"benchmark_tables_in_archive":1,"benchmark_tables_shown":1,"benchmark_tables_withheld_as_spam":0,"benchmark_definition":"a leaderboard table with at least one row; benchmark_tables_shown also counts the zero-row tables; benchmark_tables_in_archive adds the tables withheld as spam","datasets":3,"subtasks":0,"parent_tasks":1},"benchmarks":[{"leaderboard":"/sota/hyperparameter-optimization-on-bayesmark","slug":"hyperparameter-optimization-on-bayesmark","dataset":"Bayesmark","dataset_url":null,"rows_in_archive":2,"metrics":["Mean"],"first_row_in_archive_order":{"model":"HEBO","paper_title":"HEBO Pushing The Limits of Sample-Efficient Hyperparameter Optimisation","paper_url":"/paper/hebo-heteroscedastic-evolutionary-bayesian","paper_date":"2020-12-07","arxiv_id":"2012.03826","code_links":[{"title":"huawei-noah/hebo","url":"https://github.com/huawei-noah/hebo"},{"title":"huawei-noah/noah-research","url":"https://github.com/huawei-noah/noah-research/tree/master/BO/HEBO"},{"title":"huawei-noah/noah-research","url":"https://github.com/huawei-noah/noah-research/tree/master/HEBO"}],"syntology":null}}],"datasets":[{"url":"/dataset/nas-bench-201","name":"NAS-Bench-201","full_name":"","num_papers_in_archive":260},{"url":"/dataset/nas-bench-101","name":"NAS-Bench-101","full_name":"","num_papers_in_archive":152},{"url":"/dataset/pmlb","name":"PMLB","full_name":"Penn Machine Learning Benchmarks","num_papers_in_archive":42}],"subtasks":[],"parent_tasks":[{"url":"/task/automl","name":"AutoML"}],"papers":{"order":"repositories listed in the archive (desc), then date (desc); the archive holds no stars","population":"papers tagged with this task that list at least one repository in the archive","shown":30,"of":339,"tagged_in_all":813,"items":[{"url":"/paper/hyperband-a-novel-bandit-based-approach-to","title":"Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization","date":"2016-03-21","arxiv_id":"1603.06560","repositories_listed":17,"syntology":{"n":2,"n_ran":1,"n_unverified":1,"n_pointer_only":0}},{"url":"/paper/a-tutorial-on-bayesian-optimization-of","title":"A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning","date":"2010-12-12","arxiv_id":"1012.2599","repositories_listed":15,"syntology":{"n":16,"n_ran":0,"n_unverified":16,"n_pointer_only":0}},{"url":"/paper/optuna-a-next-generation-hyperparameter","title":"Optuna: A Next-generation Hyperparameter Optimization Framework","date":"2019-07-25","arxiv_id":"1907.10902","repositories_listed":11,"syntology":{"n":20,"n_ran":0,"n_unverified":20,"n_pointer_only":0}},{"url":"/paper/optimizing-millions-of-hyperparameters-by","title":"Optimizing Millions of Hyperparameters by Implicit Differentiation","date":"2019-11-06","arxiv_id":"1911.02590","repositories_listed":9,"syntology":{"n":4,"n_ran":3,"n_unverified":1,"n_pointer_only":0}},{"url":"/paper/are-gans-created-equal-a-large-scale-study","title":"Are GANs Created Equal? A Large-Scale Study","date":"2017-11-28","arxiv_id":"1711.10337","repositories_listed":9,"syntology":{"n":3,"n_ran":3,"n_unverified":0,"n_pointer_only":2}},{"url":"/paper/a-tutorial-on-bayesian-optimization","title":"A Tutorial on Bayesian Optimization","date":"2018-07-08","arxiv_id":"1807.02811","repositories_listed":7,"syntology":{"n":10,"n_ran":2,"n_unverified":8,"n_pointer_only":0}},{"url":"/paper/massively-parallel-hyperparameter-tuning","title":"A System for Massively Parallel Hyperparameter Tuning","date":"2018-10-13","arxiv_id":"1810.05934","repositories_listed":6,"syntology":{"n":15,"n_ran":0,"n_unverified":15,"n_pointer_only":0}},{"url":"/paper/optimal-hyperparameters-for-deep-lstm","title":"Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks","date":"2017-07-21","arxiv_id":"1707.06799","repositories_listed":6,"syntology":null},{"url":"/paper/gpt-takes-the-bar-exam","title":"GPT Takes the Bar Exam","date":"2022-12-29","arxiv_id":"2212.14402","repositories_listed":5,"syntology":null},{"url":"/paper/mfes-hb-efficient-hyperband-with-multi","title":"MFES-HB: Efficient Hyperband with Multi-Fidelity Quality Measurements","date":"2020-12-05","arxiv_id":"2012.03011","repositories_listed":5,"syntology":null},{"url":"/paper/single-headed-attention-rnn-stop-thinking","title":"Single Headed Attention RNN: Stop Thinking With Your Head","date":"2019-11-26","arxiv_id":"1911.11423","repositories_listed":5,"syntology":null},{"url":"/paper/nystrom-method-for-accurate-and-scalable","title":"Nystrom Method for Accurate and Scalable Implicit Differentiation","date":"2023-02-20","arxiv_id":"2302.09726","repositories_listed":4,"syntology":null},{"url":"/paper/random-search-and-reproducibility-for-neural","title":"Random Search and Reproducibility for Neural Architecture Search","date":"2019-02-20","arxiv_id":"1902.07638","repositories_listed":4,"syntology":{"n":7,"n_ran":0,"n_unverified":7,"n_pointer_only":1}},{"url":"/paper/benchmarking-automatic-machine-learning","title":"Benchmarking Automatic Machine Learning Frameworks","date":"2018-08-17","arxiv_id":"1808.06492","repositories_listed":4,"syntology":null},{"url":"/paper/tune-a-research-platform-for-distributed","title":"Tune: A Research Platform for Distributed Model Selection and Training","date":"2018-07-13","arxiv_id":"1807.05118","repositories_listed":4,"syntology":{"n":9,"n_ran":0,"n_unverified":9,"n_pointer_only":9}},{"url":"/paper/bohb-robust-and-efficient-hyperparameter","title":"BOHB: Robust and Efficient Hyperparameter Optimization at Scale","date":"2018-07-04","arxiv_id":"1807.01774","repositories_listed":4,"syntology":null},{"url":"/paper/scalable-bayesian-optimization-using-deep","title":"Scalable Bayesian Optimization Using Deep Neural Networks","date":"2015-02-19","arxiv_id":"1502.05700","repositories_listed":4,"syntology":{"n":1,"n_ran":1,"n_unverified":0,"n_pointer_only":0}},{"url":"/paper/practical-bayesian-optimization-of-machine","title":"Practical Bayesian Optimization of Machine Learning Algorithms","date":"2012-06-13","arxiv_id":"1206.2944","repositories_listed":4,"syntology":null},{"url":"/paper/lemur-neural-network-dataset-towards-seamless","title":"LEMUR Neural Network Dataset: Towards Seamless AutoML","date":"2025-04-14","arxiv_id":"2504.10552","repositories_listed":3,"syntology":null},{"url":"/paper/hyperparameter-optimization-for-randomized","title":"Hyperparameter Optimization for Randomized Algorithms: A Case Study on Random Features","date":"2024-06-30","arxiv_id":"2407.00584","repositories_listed":3,"syntology":null},{"url":"/paper/tabrepo-a-large-scale-repository-of-tabular","title":"TabRepo: A Large Scale Repository of Tabular Model Evaluations and its AutoML Applications","date":"2023-11-06","arxiv_id":"2311.02971","repositories_listed":3,"syntology":{"n":8,"n_ran":8,"n_unverified":0,"n_pointer_only":0}},{"url":"/paper/cost-effective-hyperparameter-optimization","title":"Cost-Effective Hyperparameter Optimization for Large Language Model Generation Inference","date":"2023-03-08","arxiv_id":"2303.04673","repositories_listed":3,"syntology":{"n":9,"n_ran":2,"n_unverified":7,"n_pointer_only":9}},{"url":"/paper/fednest-federated-bilevel-minimax-and","title":"FedNest: Federated Bilevel, Minimax, and Compositional Optimization","date":"2022-05-04","arxiv_id":"2205.02215","repositories_listed":3,"syntology":{"n":4,"n_ran":2,"n_unverified":2,"n_pointer_only":2}},{"url":"/paper/hebo-heteroscedastic-evolutionary-bayesian","title":"HEBO Pushing The Limits of Sample-Efficient Hyperparameter Optimisation","date":"2020-12-07","arxiv_id":"2012.03826","repositories_listed":3,"syntology":null},{"url":"/paper/model-based-asynchronous-hyperparameter","title":"Model-based Asynchronous Hyperparameter and Neural Architecture Search","date":"2020-03-24","arxiv_id":"2003.10865","repositories_listed":3,"syntology":{"n":1,"n_ran":0,"n_unverified":1,"n_pointer_only":0}},{"url":"/paper/bananas-bayesian-optimization-with-neural","title":"BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search","date":"2019-10-25","arxiv_id":"1910.11858","repositories_listed":3,"syntology":{"n":8,"n_ran":0,"n_unverified":8,"n_pointer_only":0}},{"url":"/paper/self-tuning-networks-bilevel-optimization-of","title":"Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions","date":"2019-03-07","arxiv_id":"1903.03088","repositories_listed":3,"syntology":null},{"url":"/paper/automatic-gradient-boosting","title":"Automatic Gradient Boosting","date":"2018-07-10","arxiv_id":"1807.03873","repositories_listed":3,"syntology":null},{"url":"/paper/online-learning-rate-adaptation-with","title":"Online Learning Rate Adaptation with Hypergradient Descent","date":"2017-03-14","arxiv_id":"1703.04782","repositories_listed":3,"syntology":{"n":1,"n_ran":1,"n_unverified":0,"n_pointer_only":0}},{"url":"/paper/global-optimization-of-lipschitz-functions","title":"Global optimization of Lipschitz functions","date":"2017-03-07","arxiv_id":"1703.02628","repositories_listed":3,"syntology":{"n":3,"n_ran":1,"n_unverified":2,"n_pointer_only":0}}],"syntology_records":17,"syntology_note":"a paper without a record is not a recorded non-run: it may lack an arXiv id or simply be absent from the graph layer"},"description_links":{"kept":0,"unwrapped_to_text":0,"bare_urls_linked":0,"relative_images_dropped":0,"rule":"internal links are kept only when the target slug exists in the catalog"},"syntology":{"read_at":"2026-09-24T18:15:14+00:00","claim":"Per-sample execution status on synthesized fixtures ('ran N of M samples'); not a correctness claim and not a ranking signal.","status_vocabulary":{"ran_honours":"ran, honoured the contract we drafted","ran_violates":"ran, violated the contract we drafted","ran_draft_wrong":"ran; our contract draft was wrong, not the code","ran_fixture":"ran; our fixture could not drive it","ran":"ran on a synthesized input","unverified":"unverified (harvested, no recorded run)"}},"not_shown":{"libraries":"the archive has no per-task library table","trend_sparklines":"the Trend column of the benchmarks table was a rendered image; it is not in the archive","social_and_latest_sorts":"stars and social signals are not in the archive"}}