{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/massive-exploration-of-neural-machine","title":"Massive Exploration of Neural Machine Translation Architectures","arxiv_id":"1703.03906","date":"2017-03-11","proceeding":"EMNLP 2017 9","authors":["Denny Britz","Anna Goldie","Minh-Thang Luong","Quoc Le"],"abstract":"Neural Machine Translation (NMT) has shown remarkable progress over the past\nfew years with production systems now being deployed to end-users. One major\ndrawback of current architectures is that they are expensive to train,\ntypically requiring days to weeks of GPU time to converge. This makes\nexhaustive hyperparameter search, as is commonly done with other neural network\narchitectures, prohibitively expensive. In this work, we present the first\nlarge-scale analysis of NMT architecture hyperparameters. We report empirical\nresults and variance numbers for several hundred experimental runs,\ncorresponding to over 250,000 GPU hours on the standard WMT English to German\ntranslation task. Our experiments lead to novel insights and practical advice\nfor building and extending NMT architectures. As part of this contribution, we\nrelease an open-source NMT framework that enables researchers to easily\nexperiment with novel techniques and reproduce state of the art results.","url_abs":"http://arxiv.org/abs/1703.03906v2","url_pdf":"http://arxiv.org/pdf/1703.03906v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"massive-exploration-of-neural-machine","repo_url":"https://github.com/google/seq2seq","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"massive-exploration-of-neural-machine","repo_url":"https://github.com/A-Jacobson/minimal-nmt","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"massive-exploration-of-neural-machine","repo_url":"https://github.com/AI-coder/SEQ2SEA","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"massive-exploration-of-neural-machine","repo_url":"https://github.com/ZanzaKlaus/DeepTranslate","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}},{"paper_slug":"massive-exploration-of-neural-machine","repo_url":"https://github.com/alanjia163/first1","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"massive-exploration-of-neural-machine","repo_url":"https://github.com/caesium94/seq2seq_fix","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"massive-exploration-of-neural-machine","repo_url":"https://github.com/luyige/test-git","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"massive-exploration-of-neural-machine","repo_url":"https://github.com/mahimanzum/seq2seq_corrected_for_NCT","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"massive-exploration-of-neural-machine","repo_url":"https://github.com/neqkir/attention-mechanism","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"unanswered"}},{"paper_slug":"massive-exploration-of-neural-machine","repo_url":"https://github.com/outcastofmusic/quick-nlp","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}},{"paper_slug":"massive-exploration-of-neural-machine","repo_url":"https://github.com/simonjisu/NMT","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"massive-exploration-of-neural-machine","repo_url":"https://github.com/soupstandstop/test","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"tf","reach":{"status":"gone","observed_at":"2026-09-18","how":"tree_404+repo_404"}}],"tasks":[{"task_slug":null,"task_name":"GPU"},{"task_slug":"machine-translation","task_name":"Machine Translation"},{"task_slug":"nmt","task_name":"NMT"},{"task_slug":"translation","task_name":"Translation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=1703.03906","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"1703.03906"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/simonjisu/NMT","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/neqkir/attention-mechanism","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google/seq2seq","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/alanjia163/first1","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/soupstandstop/test","reach":{"status":"gone","observed_at":"2026-09-18","how":"tree_404+repo_404"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/AI-coder/SEQ2SEA","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/luyige/test-git","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mahimanzum/seq2seq_corrected_for_NCT","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/outcastofmusic/quick-nlp","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/A-Jacobson/minimal-nmt","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/caesium94/seq2seq_fix","reach":{"status":"ok","spdx":"Apache-2.0"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ZanzaKlaus/DeepTranslate","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":1,"unverified":8},"by_repo_kind":{"official":{"samples":1,"ran":0,"repositories":1},"listed":{"samples":8,"ran":1,"repositories":3}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"3b0218c9534b6ee7","entry":"unicode_to_ascii","repo":"simonjisu/NMT","repo_kind":"listed","path":"model/preprocessing.py","file_url":"https://github.com/simonjisu/NMT/blob/HEAD/model/preprocessing.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3b0218c9534b6ee7"}},{"code_sha256_prefix":"358434439625669b","entry":"cross_entropy_sequence_loss","repo":"AI-coder/SEQ2SEA","repo_kind":"listed","path":"seq2seq/losses.py","file_url":"https://github.com/AI-coder/SEQ2SEA/blob/HEAD/seq2seq/losses.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"358434439625669b"}},{"code_sha256_prefix":"c575a1dcdbff36b2","entry":"get_dict_from_collection","repo":"google/seq2seq","repo_kind":"official","path":"seq2seq/graph_utils.py","file_url":"https://github.com/google/seq2seq/blob/HEAD/seq2seq/graph_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c575a1dcdbff36b2"}},{"code_sha256_prefix":"376ab047b1a94d61","entry":"normalize_string","repo":"simonjisu/NMT","repo_kind":"listed","path":"model/preprocessing.py","file_url":"https://github.com/simonjisu/NMT/blob/HEAD/model/preprocessing.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"376ab047b1a94d61"}},{"code_sha256_prefix":"99848e825367125b","entry":"read_files","repo":"simonjisu/NMT","repo_kind":"listed","path":"model/preprocessing.py","file_url":"https://github.com/simonjisu/NMT/blob/HEAD/model/preprocessing.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"99848e825367125b"}},{"code_sha256_prefix":"644dca570ee1eaa4","entry":"sequence_to_text","repo":"A-Jacobson/minimal-nmt","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/A-Jacobson/minimal-nmt/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"644dca570ee1eaa4"}},{"code_sha256_prefix":"575fed7c3e1f44b9","entry":"show_attention","repo":"A-Jacobson/minimal-nmt","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/A-Jacobson/minimal-nmt/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"575fed7c3e1f44b9"}},{"code_sha256_prefix":"c73e2ed8b047237f","entry":"templatemethod","repo":"AI-coder/SEQ2SEA","repo_kind":"listed","path":"seq2seq/graph_utils.py","file_url":"https://github.com/AI-coder/SEQ2SEA/blob/HEAD/seq2seq/graph_utils.py","link_basis":"plan_row","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c73e2ed8b047237f"}},{"code_sha256_prefix":"5f3f24c702dc6f63","entry":"text_to_sequence","repo":"A-Jacobson/minimal-nmt","repo_kind":"listed","path":"utils.py","file_url":"https://github.com/A-Jacobson/minimal-nmt/blob/HEAD/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5f3f24c702dc6f63"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}