Papers › Tensor2Tensor for Neural Machine Translation

Tensor2Tensor for Neural Machine Translation

16 Mar 2018WS 2018 3arXiv:1803.07416archive 2025-07-28

Ashish Vaswani, Samy Bengio, Eugene Brevdo, Francois Chollet, Aidan N. Gomez, Stephan Gouws, Llion Jones, Łukasz Kaiser, Nal Kalchbrenner, Niki Parmar, Ryan Sepassi, Noam Shazeer, Jakob Uszkoreit

Tensor2Tensor is a library for deep learning models that is well-suited for neural machine translation and includes the reference implementation of the state-of-the-art Transformer model.

PaperPDFConference PDFCodeCode Syntology ran

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, Syntology's MCP tool lists every function and class Syntology harvested from this paper and whether it ran (how to connect): get_harvested_code_for_paper(arxiv_id="1803.07416")

Code

Syntology Ran 1 of 48 code samples harvested from 12 repositories linked to this paper; 47 have no recorded run. Of those that ran: 1 ran · our draft was wrong.

By repository: official repository: 1 sample from 1 repository, 1 ran; community (archive-listed): 47 samples from 11 repositories, 0 ran. The run record, sample by sample. “Ran” means executed on a synthesized input, not that the code is correct or reproduces the paper.

15 repositories listed; official and paper-mentioned ones first.

tensorflow/tensor2tensor officialmentioned in papermentioned on GitHubtf report
1iyc/T2T-Analysis mentioned on GitHubtfApache-2.0 report
Eamin-sust/tensor2tensor mentioned on GitHubtfApache-2.0 report
Erra-Ernesto/master mentioned on GitHubtfApache-2.0 report
Janice-M/tensorflow-lessons mentioned on GitHubtfApache-2.0 report
JiachenMao/TransfornerPrune mentioned on GitHubtfApache-2.0 report
Pek20180909/dp-transformer mentioned on GitHubtfApache-2.0 report
artitw/tensor2tensor mentioned on GitHubtfApache-2.0 report
gitboyzorro5/tensor2tensor mentioned on GitHubtfApache-2.0 report
hhy37/tensor2tensor mentioned on GitHubtfApache-2.0 report
inzva/fake-academic-paper-generation mentioned on GitHubpytorch report
jinliangwei/tensor2tensor-1.11 mentioned on GitHubtfApache-2.0 report
qijimrc/tensor2tensor mentioned on GitHubtfApache-2.0 report
samsucik/t2t mentioned on GitHubtfApache-2.0 report
tianhai123/- mentioned on GitHubtfApache-2.0 report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

48 samples harvested; 1 ran; 0 honoured the contract we drafted; 47 have no recorded run. Read from Syntology's graph 2026-09-24; that is when this build read the record, not when the samples ran.

1ran · our draft was wrong
47unverified

Licence: 0 of the 48 samples are pointer only, meaning Syntology does not serve that copy's text. This page shows no code text for any sample; each one links to its file in the repository.

Harvested from 12 repositories linked to this paper, official or community; each sample names its own and says which. “Ran” means the sample executed on a synthesized input. It does not mean the output is correct, and nothing here reproduces the paper's results. “Honoured” and “violated” refer to a contract Syntology drafted from the code itself; “our draft was wrong” and “fixture could not drive it” are failures of Syntology's instrument, not of the code.

Each sample ends with its code_sha256, Syntology's identity for that exact code. An agent fetches the stored sample with Syntology's MCP tool get_code(code_sha256="…") (how to connect); click an identity to copy that call.

Repository labels, per sample. official repository: The archive marks this repository official for the paper. named in the paper: The archive records that the paper mentions this repository; it is not marked official. community (archive-listed): In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper. found in paper text by Syntology: Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted. community: Not in the archive's code links for this paper; a community repository Syntology harvested. Samples from a repository marked official are listed first. Licence labels name the repository's licence as recorded at harvest. “Pointer only” means Syntology does not serve that copy's text, for one of four reasons: no licence file was found; the licence was not identified; the licence is recorded as permissive but that copy's record is not marked cleared; or the licence is outside the permissive list Syntology serves text under (MIT, Apache-2.0, BSD and similar). Some licences outside that list permit redistribution, such as WTFPL, and GPL-3.0 under its conditions; they are simply not on the list. Hover a licence label for the reason. File links open the file on GitHub at the default branch, which may have changed since the harvest.

problem_hparams_to_features tensorflow/tensor2tensor/tensor2tensor/data_generators/problem.py official repository ran · our draft was wrong Apache-2.0 (permissive) · 4116527c2a478716 · report
add_delta_deltas Eamin-sust/tensor2tensor/tensor2tensor/layers/common_audio.py community (archive-listed) unverified Apache-2.0 (permissive) · 748914abdfd6b822 · report
add_delta_deltas Erra-Ernesto/master/tensor2tensor/layers/common_audio.py community (archive-listed) unverified Apache-2.0 (permissive) · afcb74fd5f657185 · report
add_evolved_transformer_hparams Pek20180909/dp-transformer/tensor2tensor/models/evolved_transformer.py community (archive-listed) unverified Apache-2.0 (permissive) · a39a84994bf39310 · report
add_standard_attention_hparams Erra-Ernesto/master/tensor2tensor/layers/common_attention.py community (archive-listed) unverified Apache-2.0 (permissive) · 74d5ffaeacd67288 · report
add_weight JiachenMao/TransfornerPrune/tensor2tensor/layers/bayes.py community (archive-listed) unverified Apache-2.0 (permissive) · b9294972d34bac80 · report
average_sharded_losses 1iyc/T2T-Analysis/tensor2tensor/utils/t2t_model.py community (archive-listed) unverified Apache-2.0 (permissive) · e0f4770f7b603395 · report
comma_separated_string_to_integer_list Erra-Ernesto/master/tensor2tensor/layers/common_layers.py community (archive-listed) unverified Apache-2.0 (permissive) · f5e828468ccc9543 · report
compute_mel_filterbank_features Eamin-sust/tensor2tensor/tensor2tensor/layers/common_audio.py community (archive-listed) unverified Apache-2.0 (permissive) · 0cd8646d2fafc11f · report
convert_gradient_to_tensor Erra-Ernesto/master/tensor2tensor/layers/common_layers.py community (archive-listed) unverified Apache-2.0 (permissive) · 5ea4d4b72662bfaf · report
create_host_call samsucik/t2t/tensor2tensor/utils/t2t_model.py community (archive-listed) unverified Apache-2.0 (permissive) · adbdf489f02bb809 · report
create_regularization_loss_fn Erra-Ernesto/master/tensor2tensor/layers/bayes.py community (archive-listed) unverified Apache-2.0 (permissive) · 9f7a9aa79bcd909d · report
deserialize JiachenMao/TransfornerPrune/tensor2tensor/layers/bayes.py community (archive-listed) unverified Apache-2.0 (permissive) · 8e8132bfb8eb0cd2 · report
distorted_bounding_box_crop Eamin-sust/tensor2tensor/tensor2tensor/data_generators/imagenet.py community (archive-listed) unverified Apache-2.0 (permissive) · c3b99eca8b1dea7b · report
distorted_bounding_box_crop Erra-Ernesto/master/tensor2tensor/data_generators/imagenet.py community (archive-listed) unverified Apache-2.0 (permissive) · 50ee75354b0c6168 · report
dropout_with_broadcast_dims Erra-Ernesto/master/tensor2tensor/layers/common_layers.py community (archive-listed) unverified Apache-2.0 (permissive) · 0e0b6e97db301154 · report
dropout_with_broadcast_dims qijimrc/tensor2tensor/tensor2tensor/layers/common_layers.py community (archive-listed) unverified Apache-2.0 (permissive) · 6faf23aab5900dc8 · report
encoder_decoder_attention_loss gitboyzorro5/tensor2tensor/tensor2tensor/layers/common_attention.py community (archive-listed) unverified Apache-2.0 (permissive) · 6b159f110a2bd644 · report
get JiachenMao/TransfornerPrune/tensor2tensor/layers/bayes.py community (archive-listed) unverified Apache-2.0 (permissive) · 30733be1f346d1da · report
imagenet_preprocess_example Eamin-sust/tensor2tensor/tensor2tensor/data_generators/imagenet.py community (archive-listed) unverified Apache-2.0 (permissive) · 1beeedf33db29766 · report
imagenet_preprocess_example Erra-Ernesto/master/tensor2tensor/data_generators/imagenet.py community (archive-listed) unverified Apache-2.0 (permissive) · 5393a76ac31b9d6f · report
imagenet_preprocess_example Janice-M/tensorflow-lessons/tensor2tensor/data_generators/imagenet.py community (archive-listed) unverified Apache-2.0 (permissive) · 80cb2afad98acef4 · report
imagenet_preprocess_example JiachenMao/TransfornerPrune/tensor2tensor/data_generators/imagenet.py community (archive-listed) unverified Apache-2.0 (permissive) · 512b3e2718d80431 · report
is_in_expr 1iyc/T2T-Analysis/tensor2tensor/data_generators/algorithmic_math.py community (archive-listed) unverified Apache-2.0 (permissive) · 7e2bc85a4d4189a9 · report
large_compatible_negative Eamin-sust/tensor2tensor/tensor2tensor/layers/common_attention.py community (archive-listed) unverified Apache-2.0 (permissive) · 4dce1b2c80bc7259 · report
large_compatible_negative Janice-M/tensorflow-lessons/tensor2tensor/layers/common_attention.py community (archive-listed) unverified Apache-2.0 (permissive) · 9a3f3b5c8fe9f016 · report
lower_endian_to_number 1iyc/T2T-Analysis/tensor2tensor/data_generators/algorithmic.py community (archive-listed) unverified Apache-2.0 (permissive) · b1896a2536602d28 · report
maybe_upcast Eamin-sust/tensor2tensor/tensor2tensor/layers/common_attention.py community (archive-listed) unverified Apache-2.0 (permissive) · f71eb3208ea1ac2e · report
maybe_upcast Janice-M/tensorflow-lessons/tensor2tensor/layers/common_attention.py community (archive-listed) unverified Apache-2.0 (permissive) · ca13c475bfeed81f · report
mixed_precision_is_enabled Eamin-sust/tensor2tensor/tensor2tensor/layers/common_attention.py community (archive-listed) unverified Apache-2.0 (permissive) · db9de8c21cfb33b2 · report
mixed_precision_is_enabled Janice-M/tensorflow-lessons/tensor2tensor/layers/common_attention.py community (archive-listed) unverified Apache-2.0 (permissive) · a26680f8cb87d722 · report
prepare_data 1iyc/T2T-Analysis/tensor2tensor/bin/t2t_attack.py community (archive-listed) unverified Apache-2.0 (permissive) · 2d21f2cb3f076c8e · report
preprocess_for_train Eamin-sust/tensor2tensor/tensor2tensor/data_generators/imagenet.py community (archive-listed) unverified Apache-2.0 (permissive) · d0146eb229788856 · report
preprocess_for_train Erra-Ernesto/master/tensor2tensor/data_generators/imagenet.py community (archive-listed) unverified Apache-2.0 (permissive) · c31961a8f08b93ca · report
preprocess_for_train Janice-M/tensorflow-lessons/tensor2tensor/data_generators/imagenet.py community (archive-listed) unverified Apache-2.0 (permissive) · d323135519eb2eaa · report
preprocess_for_train JiachenMao/TransfornerPrune/tensor2tensor/data_generators/imagenet.py community (archive-listed) unverified Apache-2.0 (permissive) · 2ab69b6d79c736b0 · report
project_hidden qijimrc/tensor2tensor/tensor2tensor/layers/discretization.py community (archive-listed) unverified Apache-2.0 (permissive) · 7db79e182d1bda05 · report
random_expr 1iyc/T2T-Analysis/tensor2tensor/data_generators/algorithmic_math.py community (archive-listed) unverified Apache-2.0 (permissive) · 916cc023d1d1ddbd · report
random_expr_with_required_var 1iyc/T2T-Analysis/tensor2tensor/data_generators/algorithmic_math.py community (archive-listed) unverified Apache-2.0 (permissive) · 0a76872f8ddc4b1e · report
scores_filter Eamin-sust/tensor2tensor/tensor2tensor/insights/transformer_model.py community (archive-listed) unverified Apache-2.0 (permissive) · 71927fc77962a843 · report
seq_filter Eamin-sust/tensor2tensor/tensor2tensor/insights/transformer_model.py community (archive-listed) unverified Apache-2.0 (permissive) · 6f29b141db6bc7e8 · report
shift_and_pad Pek20180909/dp-transformer/tensor2tensor/models/research/transformer_aux.py community (archive-listed) unverified Apache-2.0 (permissive) · 59c61142257c9e1e · report
slice_hidden qijimrc/tensor2tensor/tensor2tensor/layers/discretization.py community (archive-listed) unverified Apache-2.0 (permissive) · 07193e79db3b92e4 · report
smart_constant_value tianhai123/-/tensor2tensor/layers/bayes.py community (archive-listed) unverified Apache-2.0 (permissive) · 4f3643595cc37d38 · report
swap_time_and_batch_axes hhy37/tensor2tensor/tensor2tensor/layers/common_video.py community (archive-listed) unverified Apache-2.0 (permissive) · 592f83812bb35157 · report
topk_watch_fn Eamin-sust/tensor2tensor/tensor2tensor/insights/transformer_model.py community (archive-listed) unverified Apache-2.0 (permissive) · 7d5116b9da3e71ed · report
zipf_distribution 1iyc/T2T-Analysis/tensor2tensor/data_generators/algorithmic.py community (archive-listed) unverified Apache-2.0 (permissive) · 68fe9a15fc61f6bd · report
zipf_random_sample 1iyc/T2T-Analysis/tensor2tensor/data_generators/algorithmic.py community (archive-listed) unverified Apache-2.0 (permissive) · d0d6f2cc06f67920 · report

Tasks

Deep LearningMachine TranslationTranslation

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

Absolute Position EncodingsAdamAttentionBPEDense ConnectionsDropoutLabel SmoothingLayer NormalizationLinear LayerMulti-Head AttentionPosition-Wise Feed-Forward LayerReLUResidual ConnectionSoftmaxTransformer

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections