Diagnostic

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

1 dataset whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 1,213 papers with code (4,513 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Is Heuristic Sampling Necessary in Training Deep Object Detectors?
  11 Sep 2019 14 repositories listed
  In this paper, we challenge the necessity of such hard/soft sampling methods for training accurate deep object detectors.
• The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions
  28 Mar 2018 13 repositories listed Syntology ran 0 of 4 samples · 4 unverified
  Training of neural networks for automated diagnosis of pigmented skin lesions is hampered by the small size and lack of diversity of available datasets of dermatoscopic images.
• GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
  20 Apr 2018 11 repositories listed Syntology ran 5 of 14 samples · 9 unverified · 1 pointer-only (licence)
  For natural language understanding (NLU) technology to be maximally useful, both practically and as a scientific object of study, it must be general: it must be able to process language in a way that is not exclusively…
• HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models
  23 Oct 2023 9 repositories listed Syntology ran 2 of 8 samples · 6 unverified · 1 pointer-only (licence)
  Our comprehensive case studies within HallusionBench shed light on the challenges of hallucination and illusion in LVLMs.
• On the limits of cross-domain generalization in automated X-ray prediction
  6 Feb 2020 9 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 3 pointer-only (licence)
  This large scale study focuses on quantifying what X-rays diagnostic prediction tasks generalize well across multiple different datasets.
• Pareto Smoothed Importance Sampling
  9 Jul 2015 9 repositories listed Syntology ran 2 of 3 samples · 1 unverified · 3 pointer-only (licence)
  Importance weighting is a general way to adjust Monte Carlo integration to account for draws from the wrong distribution, but the resulting estimate can be highly variable when the importance ratios have a heavy right…
• NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search
  2 Jan 2020 6 repositories listed Syntology ran 6 of 20 samples · 14 unverified
  A variety of algorithms search architectures under different search space.
• Recurrent Relational Networks
  21 Nov 2017 6 repositories listed
  We achieve state of the art results on the bAbI textual question-answering dataset with the recurrent relational network, consistently solving 20/20 tasks.
• POCOVID-Net: Automatic Detection of COVID-19 From a New Lung Ultrasound Imaging Dataset (POCUS)
  25 Apr 2020 5 repositories listed
  For detecting COVID-19 in particular, the model performs with a sensitivity of 0.
• Conditional Density Estimation Tools in Python and R with Applications to Photometric Redshifts and Likelihood-Free Cosmological Inference
  30 Aug 2019 5 repositories listed Syntology ran 0 of 6 samples · 6 unverified
  We provide sample code in Python and R as well as examples of applications to photometric redshift estimation and likelihood-free cosmological inference via CDE.
• CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text
  16 Aug 2019 5 repositories listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  The recent success of natural language understanding (NLU) systems has been troubled by results highlighting the failure of these models to generalize in a systematic and robust way.
• CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
  20 Dec 2016 5 repositories listed Syntology ran 2 of 5 samples · 3 unverified · 4 pointer-only (licence)
  When building artificial intelligence systems that can reason and answer questions about visual data, we need diagnostic tests to analyze our progress and discover shortcomings.
• JANA: Jointly Amortized Neural Approximation of Complex Bayesian Models
  17 Feb 2023 4 repositories listed Syntology ran 0 of 11 samples · 11 unverified
  This work proposes ``jointly amortized neural approximation'' (JANA) of intractable likelihood functions and posterior densities arising in Bayesian surrogate modeling and simulation-based inference.
• Vision-and-Language or Vision-for-Language? On Cross-Modal Influence in Multimodal Transformers
  9 Sep 2021 4 repositories listed Syntology ran 0 of 7 samples · 7 unverified
  Models that have learned to construct cross-modal representations using both modalities are expected to perform worse when inputs are missing from a modality.
• NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Size
  28 Aug 2020 4 repositories listed Syntology ran 4 of 4 samples · 0 unverified · 2 pointer-only (licence)
  In this paper, we propose NATS-Bench, a unified benchmark on searching for both topology and size, for (almost) any up-to-date NAS algorithm.
• Exploring Automatic Diagnosis of COVID-19 from Crowdsourced Respiratory Sound Data
  10 Jun 2020 4 repositories listed
  This work opens the door to further investigation of how automatically analysed respiratory patterns could be used as pre-screening signals to aid COVID-19 diagnosis.
• Evaluation of Retinal Image Quality Assessment Networks in Different Color-spaces
  10 Jul 2019 4 repositories listed
  Retinal image quality assessment (RIQA) is essential for controlling the quality of retinal imaging and guaranteeing the reliability of diagnoses by ophthalmologists or automated analysis systems.
• auditor: an R Package for Model-Agnostic Visual Validation and Diagnostics
  19 Sep 2018 4 repositories listed
  With modern software it is easy to train even a~complex model that fits the training data and results in high accuracy on the test set.
• Differentiable Voxel-based X-ray Rendering Improves Sparse-View 3D CBCT Reconstruction
  28 Nov 2024 3 repositories listed Syntology ran 2 of 17 samples · 15 unverified
  We present DiffVox, a self-supervised framework for Cone-Beam Computed Tomography (CBCT) reconstruction by directly optimizing a voxelgrid representation using physics-based differentiable X-ray rendering.
• RaTEScore: A Metric for Radiology Report Generation
  24 Jun 2024 3 repositories listed
  This paper introduces a novel, entity-aware metric, termed as Radiological Report (Text) Evaluation (RaTEScore), to assess the quality of medical reports generated by AI models.
• Zero-1-to-3: Domain-level Zero-shot Cognitive Diagnosis via One Batch of Early-bird Students towards Three Diagnostic Objectives
  20 Dec 2023 3 repositories listed
  Consequently, we refine the cognitive states of cold-start students as diagnostic outcomes via virtual data, aligning with the diagnosis-oriented goal.
• MVBench: A Comprehensive Multi-modal Video Understanding Benchmark
  28 Nov 2023 3 repositories listed Syntology ran 7 of 10 samples · 3 unverified
  With the rapid development of Multi-modal Large Language Models (MLLMs), a number of diagnostic benchmarks have recently emerged to evaluate the comprehension capabilities of these models.
• BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Models
  12 Sep 2023 3 repositories listed
  As GPT-4 is purportedly one of the best-performing multilingual LLMs at the moment, we use it as a yardstick to gauge the capabilities of LLMs in the context of SEA languages.
• Augmenting Chest X-ray Datasets with Non-Expert Annotations
  5 Sep 2023 3 repositories listed
  Another strategy to increase the size of a dataset is crowdsourcing, a widely adopted practice in general computer vision with some success in medical image analysis.
• Segment Anything in Medical Images
  24 Apr 2023 3 repositories listed Syntology ran 6 of 14 samples · 8 unverified · 5 pointer-only (licence)
  Medical image segmentation is a critical component in clinical practice, facilitating accurate diagnosis, treatment planning, and disease monitoring.
• Robust and Efficient Medical Imaging with Self-Supervision
  19 May 2022 3 repositories listed Syntology ran 5 of 28 samples · 23 unverified
  These results suggest that REMEDIS can significantly accelerate the life-cycle of medical imaging AI development thereby presenting an important step forward for medical imaging AI to deliver broad impact.
• Online Forecasting and Anomaly Detection Based on the ARIMA Model
  2 Apr 2021 3 repositories listed
  Real-time diagnostics of complex technical systems such as power plants are critical to keep the system in its working state.
• HateCheck: Functional Tests for Hate Speech Detection Models
  31 Dec 2020 3 repositories listed
  Detecting online hate is a difficult task that even state-of-the-art models struggle with.
• COVID-CT-MD: COVID-19 Computed Tomography (CT) Scan Dataset Applicable in Machine Learning and Deep Learning
  28 Sep 2020 3 repositories listed
  Novel Coronavirus (COVID-19) has drastically overwhelmed more than 200 countries affecting millions and claiming almost 1 million lives, since its emergence in late 2019.
• Generalized Word Shift Graphs: A Method for Visualizing and Explaining Pairwise Comparisons Between Texts
  5 Aug 2020 3 repositories listed
  A common task in computational text analyses is to quantify how two corpora differ according to a measurement like word frequency, sentiment, or information content.

Syntology lines on 16 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.