{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/optimal-hyperparameters-for-deep-lstm","title":"Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks","arxiv_id":"1707.06799","date":"2017-07-21","proceeding":null,"authors":["Nils Reimers","Iryna Gurevych"],"abstract":"Selecting optimal parameters for a neural network architecture can often make\nthe difference between mediocre and state-of-the-art performance. However,\nlittle is published which parameters and design choices should be evaluated or\nselected making the correct hyperparameter optimization often a \"black art that\nrequires expert experiences\" (Snoek et al., 2012). In this paper, we evaluate\nthe importance of different network design choices and hyperparameters for five\ncommon linguistic sequence tagging tasks (POS, Chunking, NER, Entity\nRecognition, and Event Detection). We evaluated over 50.000 different setups\nand found, that some parameters, like the pre-trained word embeddings or the\nlast layer of the network, have a large impact on the performance, while other\nparameters, for example the number of LSTM layers or the number of recurrent\nunits, are of minor importance. We give a recommendation on a configuration\nthat performs well among different tasks.","url_abs":"http://arxiv.org/abs/1707.06799v2","url_pdf":"http://arxiv.org/pdf/1707.06799v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"optimal-hyperparameters-for-deep-lstm","repo_url":"https://github.com/UKPLab/emnlp2017-bilstm-cnn-crf","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"optimal-hyperparameters-for-deep-lstm","repo_url":"https://github.com/SuphanutN/Thai-NER-BiLSTM-WordCharEmbedding","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"optimal-hyperparameters-for-deep-lstm","repo_url":"https://github.com/SuphanutN/Thai-NER-BiLSTMCRF-WordCharEmbedding","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"unanswered"}},{"paper_slug":"optimal-hyperparameters-for-deep-lstm","repo_url":"https://github.com/jiesutd/NCRFpp","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"optimal-hyperparameters-for-deep-lstm","repo_url":"https://github.com/jiesutd/PyTorchSeqLabel","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"optimal-hyperparameters-for-deep-lstm","repo_url":"https://github.com/jiesutd/yato","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"chunking","task_name":"Chunking"},{"task_slug":"event-detection","task_name":"Event Detection"},{"task_slug":"hyperparameter-optimization","task_name":"Hyperparameter Optimization"},{"task_slug":"named-entity-recognition-ner","task_name":"Named Entity Recognition (NER)"},{"task_slug":"pos","task_name":"POS"},{"task_slug":"word-embeddings","task_name":"Word Embeddings"}],"methods":[{"method_slug":"lstm","method_name":"LSTM"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"tanh-activation","method_name":"Tanh Activation"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1707.06799","mcp":null,"developers":"https://syntology.ai/developers"},"arxiv_metadata":null,"syntology_extracted_results":null}