Experimental Design

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

No dataset record in the archive lists this task.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 197 papers with code (688 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• OpenBox: A Generalized Black-box Optimization Service
  1 Jun 2021 6 repositories listed
  Black-box optimization (BBO) has a broad range of applications, including automatic machine learning, engineering, physics, and experimental design.
• Probing the limitations of multimodal language models for chemistry and materials research
  25 Nov 2024 3 repositories listed Syntology ran 0 of 9 samples · 9 unverified
  Recent advancements in artificial intelligence have sparked interest in scientific assistants that could support researchers across the full spectrum of scientific workflows, from literature review to experimental…
• Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers
  6 Sep 2024 3 repositories listed Syntology ran 0 of 5 samples · 5 unverified
  Recent advancements in large language models (LLMs) have sparked optimism about their potential to accelerate scientific discovery, with a growing number of works proposing research agents that autonomously generate and…
• Better than classical? The subtle art of benchmarking quantum machine learning models
  11 Mar 2024 3 repositories listed Syntology ran 2 of 2 samples · 0 unverified
  Benchmarking models via classical simulations is one of the main ways to judge ideas in quantum machine learning before noise-free hardware is available.
• Empirical Study of Off-Policy Policy Evaluation for Reinforcement Learning
  15 Nov 2019 3 repositories listed
  We offer an experimental benchmark and empirical study for off-policy policy evaluation (OPE) in reinforcement learning, which is a key problem in many safety critical applications.
• Optimising Neural Fractional Differential Equations for Performance and Efficiency
  20 Oct 2024 2 repositories listed
  Neural Ordinary Differential Equations (NODEs) are well-established architectures that fit an ODE, modelled by a neural network (NN), to data, effectively modelling complex dynamical systems.
• Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent System
  12 Oct 2024 2 repositories listed Syntology ran 3 of 3 samples · 0 unverified
  The rapid advancement of scientific progress requires innovative tools that can accelerate knowledge discovery.
• GitHub is an effective platform for collaborative and reproducible laboratory research
  18 Aug 2024 2 repositories listed
  Laboratory research is a complex, collaborative process that involves several stages, including hypothesis formulation, experimental design, data generation and analysis, and manuscript writing.
• NNsight and NDIF: Democratizing Access to Open-Weight Foundation Model Internals
  18 Jul 2024 2 repositories listed Syntology ran 0 of 8 samples · 8 unverified
  We introduce NNsight and NDIF, technologies that work in tandem to enable scientific study of the representations and computations learned by very large neural networks.
• Variance Alignment Score: A Simple But Tough-to-Beat Data Selection Method for Multimodal Contrastive Learning
  3 Feb 2024 2 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 3 pointer-only (licence)
  In recent years, data selection has emerged as a core issue for large-scale visual-language model pretraining, especially on noisy web-curated datasets.
• Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine
  28 Nov 2023 2 repositories listed Syntology ran 0 of 10 samples · 10 unverified
  We find that prompting innovation can unlock deeper specialist capabilities and show that GPT-4 easily tops prior leading results for medical benchmarks.
• An Experimental Design for Anytime-Valid Causal Inference on Multi-Armed Bandits
  9 Nov 2023 2 repositories listed
  This paper addresses both challenges by introducing the Mixture Adaptive Design (MAD), a new experimental design for multi-armed bandit (MAB) algorithms that enables anytime-valid inference on the Average Treatment…
• The Machine Psychology of Cooperation: Can GPT models operationalise prompts for altruism, cooperation, competitiveness and selfishness in economic games?
  13 May 2023 2 repositories listed Syntology ran 0 of 3 samples · 3 unverified
  Using a within-subject experimental design, we used a prompt to describe the task environment using a similar protocol to that used in experimental psychology studies with human subjects.
• Derivative-Informed Neural Operator: An Efficient Framework for High-Dimensional Parametric Derivative Learning
  21 Jun 2022 2 repositories listed
  We propose derivative-informed neural operators (DINOs), a general family of neural networks to approximate operators as infinite-dimensional mappings from input function spaces to output function spaces or quantities…
• VICE: Variational Interpretable Concept Embeddings
  2 May 2022 2 repositories listed Syntology ran 7 of 10 samples · 3 unverified · 10 pointer-only (licence)
  This paper introduces Variational Interpretable Concept Embeddings (VICE), an approximate Bayesian method for embedding object concepts in a vector space using data collected from humans in a triplet odd-one-out task.
• Learning High-Dimensional Parametric Maps via Reduced Basis Adaptive Residual Networks
  14 Dec 2021 2 repositories listed
  We propose a scalable framework for the learning of high-dimensional parametric maps via adaptively constructed residual network (ResNet) maps between reduced bases of the inputs and outputs.
• An Experimental Design Perspective on Model-Based Reinforcement Learning
  9 Dec 2021 2 repositories listed Syntology ran 4 of 4 samples · 0 unverified
  In particular, we leverage ideas from Bayesian optimal experimental design to guide the selection of state-action queries for efficient learning.
• Emulation of physical processes with Emukit
  25 Oct 2021 2 repositories listed
  Decision making in uncertain scenarios is an ubiquitous challenge in real world systems.
• GeneDisco: A Benchmark for Experimental Design in Drug Discovery
  22 Oct 2021 2 repositories listed Syntology ran 0 of 1 samples · 1 unverified
  GeneDisco contains a curated set of multiple publicly available experimental data sets as well as open-source implementations of state-of-the-art active learning policies for experimental design and exploration.
• A-Optimal Active Learning
  18 Oct 2021 2 repositories listed
  The first is based on a Bayesian interpretation of the semi-supervised learning problem with the graph Laplacian that is used for the prior distribution and the second is based on a frequentist approach, that updates…
• Edge Proposal Sets for Link Prediction
  30 Jun 2021 2 repositories listed
  Here, we demonstrate how simply adding a set of edges, which we call a \emph{proposal set}, to the graph as a pre-processing step can improve the performance of several link prediction algorithms.
• hls4ml: An Open-Source Codesign Workflow to Empower Scientific Low-Power Machine Learning Devices
  9 Mar 2021 2 repositories listed
  Accessible machine learning algorithms, software, and diagnostic tools for energy-efficient devices and systems are extremely valuable across a broad range of application domains.
• Experimental design for MRI by greedy policy search
  30 Oct 2020 2 repositories listed Syntology ran 4 of 18 samples · 14 unverified
  In today's clinical practice, magnetic resonance imaging (MRI) is routinely accelerated through subsampling of the associated Fourier domain.
• With Little Power Comes Great Responsibility
  13 Oct 2020 2 repositories listed
  Despite its importance to experimental design, statistical power (the probability that, given a real effect, an experiment will reject the null hypothesis) has largely been ignored by the NLP community.
• BoTorch: A Framework for Efficient Monte-Carlo Bayesian Optimization
  14 Oct 2019 2 repositories listed
  Bayesian optimization provides sample-efficient global optimization for a broad range of applications, including automatic machine learning, engineering, physics, and experimental design.
• Attention is not not Explanation
  13 Aug 2019 2 repositories listed
  We show that even when reliable adversarial distributions can be found, they don't perform well on the simple diagnostic, indicating that prior work does not disprove the usefulness of attention mechanisms for…
• Optimal design of experiments to identify latent behavioral types
  10 Jul 2018 2 repositories listed
  We apply our procedure to a game of imperfect information to evaluate and quantify the computational improvements.
• Creation and analysis of biochemical constraint-based models: the COBRA Toolbox v3.0
  11 Oct 2017 2 repositories listed
  This protocol can be adapted for the generation and analysis of a constraint-based model in a wide variety of molecular systems biology scenarios.
• Pitfalls and Best Practices in Algorithm Configuration
  17 May 2017 2 repositories listed
  Good parameter settings are crucial to achieve high performance in many areas of artificial intelligence (AI), such as propositional satisfiability solving, AI planning, scheduling, and machine learning (in particular…
• Learning a nonlinear dynamical system model of gene regulation: A perturbed steady-state approach
  13 Jul 2012 2 repositories listed
  Model-based methods founded on quantitative descriptions of gene regulation are among the most promising, but many such methods rely on simple, local models or on ad hoc inference approaches lacking experimental…

Syntology lines on 12 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.