{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/tpugraphs-a-performance-prediction-dataset-on-1","title":"TpuGraphs: A Performance Prediction Dataset on Large Tensor Computational Graphs","arxiv_id":"2308.13490","date":"2023-08-25","proceeding":"NeurIPS 2023 11","authors":["Phitchaya Mangpo Phothilimthana","Sami Abu-El-Haija","Kaidi Cao","Bahare Fatemi","Mike Burrows","Charith Mendis","Bryan Perozzi"],"abstract":"Precise hardware performance models play a crucial role in code optimizations. They can assist compilers in making heuristic decisions or aid autotuners in identifying the optimal configuration for a given program. For example, the autotuner for XLA, a machine learning compiler, discovered 10-20% speedup on state-of-the-art models serving substantial production traffic at Google. Although there exist a few datasets for program performance prediction, they target small sub-programs such as basic blocks or kernels. This paper introduces TpuGraphs, a performance prediction dataset on full tensor programs, represented as computational graphs, running on Tensor Processing Units (TPUs). Each graph in the dataset represents the main computation of a machine learning workload, e.g., a training epoch or an inference step. Each data sample contains a computational graph, a compilation configuration, and the execution time of the graph when compiled with the configuration. The graphs in the dataset are collected from open-source machine learning programs, featuring popular model architectures, e.g., ResNet, EfficientNet, Mask R-CNN, and Transformer. TpuGraphs provides 25x more graphs than the largest graph property prediction dataset (with comparable graph sizes), and 770x larger graphs on average compared to existing performance prediction datasets on machine learning programs. This graph-level prediction task on large graphs introduces new challenges in learning, ranging from scalability, training efficiency, to model quality.","url_abs":"https://arxiv.org/abs/2308.13490v3","url_pdf":"https://arxiv.org/pdf/2308.13490v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"tpugraphs-a-performance-prediction-dataset-on-1","repo_url":"https://github.com/google-research-datasets/tpu_graphs","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"tf","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"graph-property-prediction","task_name":"Graph Property Prediction"},{"task_slug":"prediction","task_name":"Prediction"},{"task_slug":"property-prediction","task_name":"Property Prediction"},{"task_slug":"runtime-ranking","task_name":"Runtime ranking"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"absolute-position-encodings","method_name":"Absolute Position Encodings"},{"method_slug":"adam","method_name":"Adam"},{"method_slug":"attention","method_name":"Attention"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"bpe","method_name":"BPE"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"dense-connections","method_name":"Dense Connections"},{"method_slug":"depthwise-convolution","method_name":"Depthwise Convolution"},{"method_slug":"depthwise-separable-convolution","method_name":"Depthwise Separable Convolution"},{"method_slug":"dropout","method_name":"Dropout"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"inverted-residual-block","method_name":"Inverted Residual Block"},{"method_slug":"kaiming-initialization","method_name":"Kaiming Initialization"},{"method_slug":"label-smoothing","method_name":"Label Smoothing"},{"method_slug":"layer-normalization","method_name":"Layer Normalization"},{"method_slug":"linear-layer","method_name":"Linear Layer"},{"method_slug":"mask-r-cnn","method_name":"Mask R-CNN"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"multi-head-attention","method_name":"Multi-Head Attention"},{"method_slug":"pointwise-convolution","method_name":"Pointwise Convolution"},{"method_slug":"position-wise-feed-forward-layer","method_name":"Position-Wise Feed-Forward Layer"},{"method_slug":"rmsprop","method_name":"RMSProp"},{"method_slug":"rpn","method_name":"RPN"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"},{"method_slug":"roi-align","method_name":"RoIAlign"},{"method_slug":"sigmoid-activation","method_name":"Sigmoid Activation"},{"method_slug":"softmax","method_name":"Softmax"},{"method_slug":"squeeze-and-excitation-block","method_name":"Squeeze-and-Excitation Block"},{"method_slug":"transformer","method_name":"Transformer"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/runtime-ranking-on-tpugraphs-layout-mean","task":"Runtime ranking","dataset":"TpuGraphs Layout mean","model":"TpuGraphs","rank_in_archive_order":2,"of":2,"metrics":{"Kendall's Tau":"0.298"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2308.13490","atlas_url":"https://app.syntology.ai/?focus=2308.13490","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.13490"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/google-research-datasets/tpu_graphs","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":2,"unverified":2},"by_repo_kind":{"official":{"samples":4,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"1e10d9728e25d050","entry":"random_prediction","repo":"google-research-datasets/tpu_graphs","repo_kind":"official","path":"tpu_graphs/baselines/tiles/random_baseline.py","file_url":"https://github.com/google-research-datasets/tpu_graphs/blob/HEAD/tpu_graphs/baselines/tiles/random_baseline.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1e10d9728e25d050"}},{"code_sha256_prefix":"c8a97da4a2f4ae48","entry":"shape","repo":"google-research-datasets/tpu_graphs","repo_kind":"official","path":"tpu_graphs/baselines/tiles/implicit.py","file_url":"https://github.com/google-research-datasets/tpu_graphs/blob/HEAD/tpu_graphs/baselines/tiles/implicit.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c8a97da4a2f4ae48"}},{"code_sha256_prefix":"3f00fef85db7c3ce","entry":"get_eval_indices","repo":"google-research-datasets/tpu_graphs","repo_kind":"official","path":"tpu_graphs/baselines/layout/eval_indices/validation.py","file_url":"https://github.com/google-research-datasets/tpu_graphs/blob/HEAD/tpu_graphs/baselines/layout/eval_indices/validation.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3f00fef85db7c3ce"}},{"code_sha256_prefix":"69f65974b8d50e3c","entry":"top_error_performance","repo":"google-research-datasets/tpu_graphs","repo_kind":"official","path":"tpu_graphs/baselines/tiles/metrics.py","file_url":"https://github.com/google-research-datasets/tpu_graphs/blob/HEAD/tpu_graphs/baselines/tiles/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"69f65974b8d50e3c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}