BIG-bench Machine Learning

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

2 leaderboard tables shown for this task, 2 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

1 dataset whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 2,352 papers with code (10,033 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• YOLOv4: Optimal Speed and Accuracy of Object Detection
  23 Apr 2020 223 repositories listed Syntology ran 24 of 184 samples · 160 unverified · 8 pointer-only (licence)
  There are a huge number of features which are said to improve Convolutional Neural Network (CNN) accuracy.
• UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
  9 Feb 2018 39 repositories listed
  UMAP (Uniform Manifold Approximation and Projection) is a novel manifold learning technique for dimension reduction.
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
  25 Aug 2017 37 repositories listed Syntology ran 1 of 1 samples · 0 unverified
  We present Fashion-MNIST, a new dataset comprising of 28x28 grayscale images of 70, 000 fashion products from 10 categories, with 7, 000 images per category.
• Density estimation using Real NVP
  27 May 2016 35 repositories listed Syntology ran 43 of 70 samples · 27 unverified · 33 pointer-only (licence)
  Unsupervised learning of probabilistic models is a central yet challenging problem in machine learning.
• XGBoost: A Scalable Tree Boosting System
  9 Mar 2016 28 repositories listed Syntology ran 0 of 17 samples · 17 unverified
  In this paper, we describe a scalable end-to-end tree boosting system called XGBoost, which is used widely by data scientists to achieve state-of-the-art results on many machine learning challenges.
• PennyLane: Automatic differentiation of hybrid quantum-classical computations
  12 Nov 2018 27 repositories listed Syntology ran 0 of 25 samples · 25 unverified
  PennyLane's core feature is the ability to compute gradients of variational quantum circuits in a way that is compatible with classical techniques such as backpropagation.
• Deep Learning with Differential Privacy
  1 Jul 2016 26 repositories listed Syntology ran 8 of 24 samples · 16 unverified · 3 pointer-only (licence)
  Machine learning techniques based on neural networks are achieving remarkable results in a wide variety of domains.
• Datasheets for Datasets
  23 Mar 2018 25 repositories listed Syntology ran 1 of 3 samples · 2 unverified
  The machine learning community currently has no standardized process for documenting datasets, which can lead to severe consequences in high-stakes domains.
• Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
  13 Jun 2015 23 repositories listed Syntology ran 1 of 3 samples · 2 unverified · 2 pointer-only (licence)
  The goal of precipitation nowcasting is to predict the future rainfall intensity in a local region over a relatively short period of time.
• Noise2Noise: Learning Image Restoration without Clean Data
  12 Mar 2018 21 repositories listed Syntology ran 1 of 7 samples · 6 unverified · 1 pointer-only (licence)
  We apply basic statistical reasoning to signal reconstruction by machine learning -- learning to map corrupted observations to clean signals -- with a simple and powerful conclusion: it is possible to learn to restore…
• Facial Key Points Detection using Deep Convolutional Neural Network - NaimishNet
  3 Oct 2017 19 repositories listed
  It involves predicting the co-ordinates of the FKPs, e.
• Practical Black-Box Attacks against Machine Learning
  8 Feb 2016 18 repositories listed Syntology ran 4 of 11 samples · 7 unverified · 4 pointer-only (licence)
  Our attack strategy consists in training a local model to substitute for the target DNN, using inputs synthetically generated by an adversary and labeled by the target DNN.
• Snips Voice Platform: an embedded Spoken Language Understanding system for private-by-design voice interfaces
  25 May 2018 16 repositories listed Syntology ran 1 of 15 samples · 14 unverified
  This paper presents the machine learning architecture of the Snips Voice Platform, a software solution to perform Spoken Language Understanding on microprocessors typical of IoT devices.
• fastMRI: An Open Dataset and Benchmarks for Accelerated MRI
  21 Nov 2018 13 repositories listed Syntology ran 2 of 5 samples · 3 unverified
  Accelerating Magnetic Resonance Imaging (MRI) by taking fewer measurements has the potential to reduce medical costs, minimize stress to patients and make MRI possible in applications where it is currently prohibitively…
• The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions
  28 Mar 2018 13 repositories listed Syntology ran 0 of 4 samples · 4 unverified
  Training of neural networks for automated diagnosis of pigmented skin lesions is hampered by the small size and lack of diversity of available datasets of dermatoscopic images.
• Self-Normalizing Neural Networks
  8 Jun 2017 13 repositories listed Syntology ran 1 of 1 samples · 0 unverified
  We introduce self-normalizing neural networks (SNNs) to enable high-level abstract representations.
• GNNExplainer: Generating Explanations for Graph Neural Networks
  10 Mar 2019 12 repositories listed Syntology ran 1 of 22 samples · 21 unverified
  We formulate GNNExplainer as an optimization task that maximizes the mutual information between a GNN's prediction and distribution of possible subgraph structures.
• Model Cards for Model Reporting
  5 Oct 2018 12 repositories listed Syntology ran 1 of 9 samples · 8 unverified · 1 pointer-only (licence)
  Model cards also disclose the context in which models are intended to be used, details of the performance evaluation procedures, and other relevant information.
• Challenges in Representation Learning: A report on three machine learning contests
  1 Jul 2013 12 repositories listed Syntology ran 0 of 12 samples · 12 unverified
  The ICML 2013 Workshop on Challenges in Representation Learning focused on three challenges: the black box learning challenge, the facial expression recognition challenge, and the multimodal learning challenge.
• BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
  22 Aug 2017 11 repositories listed Syntology ran 6 of 8 samples · 2 unverified · 8 pointer-only (licence)
  These results demonstrate that backdoors in neural networks are both powerful and---because the behavior of neural networks is difficult to explicate---stealthy.
• Multitask learning and benchmarking with clinical time series data
  22 Mar 2017 11 repositories listed Syntology ran 4 of 40 samples · 36 unverified · 5 pointer-only (licence)
  Health care is one of the most exciting frontiers in data mining and machine learning.
• Membership Inference Attacks against Machine Learning Models
  18 Oct 2016 11 repositories listed Syntology ran 2 of 14 samples · 12 unverified
  We quantitatively investigate how machine learning models leak information about the individual data records on which they were trained.
• Deep Learning for Classical Japanese Literature
  3 Dec 2018 10 repositories listed Syntology ran 2 of 11 samples · 9 unverified · 1 pointer-only (licence)
  Much of machine learning research focuses on producing models which perform well on benchmark tasks, in turn improving our understanding of the challenges associated with those tasks.
• Universal Differential Equations for Scientific Machine Learning
  13 Jan 2020 9 repositories listed Syntology ran 0 of 4 samples · 4 unverified
  In the context of science, the well-known adage "a picture is worth a thousand words" might well be "a model is worth a thousand datasets."
• Advances and Open Problems in Federated Learning
  10 Dec 2019 9 repositories listed Syntology ran 0 of 9 samples · 9 unverified
  FL embodies the principles of focused data collection and minimization, and can mitigate many of the systemic privacy risks and costs resulting from traditional, centralized machine learning and data science approaches.
• Neural Additive Models: Interpretable Machine Learning with Neural Nets
  29 Apr 2020 8 repositories listed Syntology ran 26 of 33 samples · 7 unverified · 14 pointer-only (licence)
  They perform similarly to existing state-of-the-art generalized additive models in accuracy, but are more flexible because they are based on neural nets instead of boosted trees.
• The Machine Learning Bazaar: Harnessing the ML Ecosystem for Effective System Development
  22 May 2019 8 repositories listed
  To address these problems, we introduce the Machine Learning Bazaar, a new framework for developing machine learning and automated machine learning software systems.
• Consistent Individualized Feature Attribution for Tree Ensembles
  12 Feb 2018 8 repositories listed Syntology ran 3 of 4 samples · 1 unverified
  A unified approach to explain the output of any machine learning model.
• Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
  21 Jul 2016 8 repositories listed Syntology ran 0 of 8 samples · 8 unverified
  Geometrically, gender bias is first shown to be captured by a direction in the word embedding.
• CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
  9 Feb 2021 7 repositories listed Syntology ran 1 of 1 samples · 0 unverified
  Benchmark datasets have a significant impact on accelerating research in programming language tasks.

Syntology lines on 27 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.