Fairness

Benchmarks archive 2025-07-28

9 leaderboard tables shown for this task, 9 with rows (a “benchmark” on this site is a table with at least one row, as on /sota), ordered by row count. “Best model” is the first row in the archive's own order at snapshot; nothing is re-ranked here and metric direction is not recorded in the archive. PwC's Trend sparklines are not in the archive, so that column is omitted.

Syntology column: samples harvested from the paper's repositories and executed on synthesized fixtures; “ran” is not a correctness claim and does not order the table. A dash means no Syntology record for that paper, not a recorded non-run. Read from the graph 2026-09-24.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

23 datasets whose archive record lists this task, ordered by the archive's paper count.

Subtasks archive 2025-07-28

1 subtask in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 1,714 papers with code (5,676 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking
  4 Apr 2020 33 repositories listed Syntology ran 8 of 53 samples · 45 unverified
  Formulating MOT as multi-task learning of object detection and re-ID in a single network is appealing since it allows joint optimization of the two tasks and enjoys high computation efficiency.
• AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias
  3 Oct 2018 13 repositories listed Syntology ran 3 of 11 samples · 8 unverified · 1 pointer-only (licence)
  Such architectural design and abstractions enable researchers and developers to extend the toolkit with their new algorithms and improvements, and to use it for performance benchmarking.
• ELEVATER: A Benchmark and Toolkit for Evaluating Language-Augmented Visual Models
  19 Apr 2022 9 repositories listed Syntology ran 4 of 20 samples · 16 unverified
  In general, these language-augmented visual models demonstrate strong transferability to a variety of datasets and tasks.
• A Critic Evaluation of Methods for COVID-19 Automatic Detection from X-Ray Images
  27 Apr 2020 9 repositories listed
  In this paper, we compare and evaluate different testing protocols used for automatic COVID-19 diagnosis from X-Ray images in the recent literature.
• Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks
  3 Oct 2019 9 repositories listed Syntology ran 3 of 13 samples · 10 unverified · 2 pointer-only (licence)
  Recently, increasing attention has been drawn to the internal mechanisms of convolutional neural networks, and the reason why the network makes specific decisions.
• Agnostic Federated Learning
  1 Feb 2019 7 repositories listed Syntology ran 3 of 10 samples · 7 unverified
  A key learning scenario in large-scale applications is that of federated learning, where a centralized model is trained based on data originating from a large number of clients.
• Learning Adversarially Fair and Transferable Representations
  17 Feb 2018 7 repositories listed Syntology ran 3 of 3 samples · 0 unverified · 3 pointer-only (licence)
  In this paper, we advocate for representation learning as the key to mitigating unfair prediction outcomes downstream.
• Analyzing Fairness in Deepfake Detection With Massively Annotated Databases
  11 Aug 2022 6 repositories listed
  In this work, we investigate factors causing biased detection in public Deepfake datasets by (a) creating large-scale demographic and non-demographic attribute annotations with 47 different attributes for five popular…
• FreeMatch: Self-adaptive Thresholding for Semi-supervised Learning
  15 May 2022 6 repositories listed Syntology ran 4 of 14 samples · 10 unverified · 2 pointer-only (licence)
  Semi-supervised Learning (SSL) has witnessed great success owing to the impressive performances brought by various methods based on pseudo labeling and consistency regularization.
• Federated Multi-Task Learning under a Mixture of Distributions
  23 Aug 2021 5 repositories listed Syntology ran 3 of 17 samples · 14 unverified · 1 pointer-only (licence)
  The increasing size of data generated by smartphones and IoT devices motivated the development of Federated Learning (FL), a framework for on-device collaborative training of machine learning models.
• Retiring Adult: New Datasets for Fair Machine Learning
  10 Aug 2021 5 repositories listed Syntology ran 3 of 3 samples · 0 unverified
  Our primary contribution is a suite of new datasets derived from US Census surveys that extend the existing data ecosystem for research on fair machine learning.
• Fairness without Demographics through Adversarially Reweighted Learning
  23 Jun 2020 5 repositories listed Syntology ran 6 of 8 samples · 2 unverified
  Much of the previous machine learning (ML) fairness literature assumes that protected features such as race and sex are present in the dataset, and relies upon them to mitigate fairness concerns.
• Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification
  11 Mar 2019 5 repositories listed
  Unintended bias in Machine Learning can manifest as systemic differences in performance for different demographic groups, potentially compounding existing challenges to fairness in society at large.
• An Empirical Study of Rich Subgroup Fairness for Machine Learning
  24 Aug 2018 5 repositories listed Syntology ran 2 of 11 samples · 9 unverified · 4 pointer-only (licence)
  In this paper, we undertake an extensive empirical evaluation of the algorithm of Kearns et al.
• Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness
  14 Nov 2017 5 repositories listed Syntology ran 2 of 4 samples · 2 unverified · 4 pointer-only (licence)
  We prove that the computational problem of auditing subgroup fairness for both equality of false positive rates and statistical parity is equivalent to the problem of weak agnostic learning, which means it is…
• Learning to Pivot with Adversarial Networks
  3 Nov 2016 5 repositories listed
  Several techniques for domain adaptation have been proposed to account for differences in the distribution of the data used for training and testing.
• GPFL: Simultaneously Learning Global and Personalized Feature Information for Personalized Federated Learning
  20 Aug 2023 4 repositories listed Syntology ran 9 of 17 samples · 8 unverified · 7 pointer-only (licence)
  Federated Learning (FL) is popular for its privacy-preserving and collaborative learning capabilities.
• Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Stochastic Approach
  23 Oct 2022 4 repositories listed Syntology ran 0 of 9 samples · 9 unverified
  Machine learning problems with multiple objective functions appear either in learning with multiple criteria where learning has to make a trade-off between multiple performance metrics such as fairness, safety and…
• Ditto: Fair and Robust Federated Learning Through Personalization
  8 Dec 2020 4 repositories listed Syntology ran 15 of 18 samples · 3 unverified · 1 pointer-only (licence)
  Fairness and robustness are two important concerns for federated learning systems.
• Learning to Continuously Optimize Wireless Resource In Episodically Dynamic Environment
  16 Nov 2020 4 repositories listed
  We propose to build the notion of continual learning (CL) into the modeling process of learning wireless systems, so that the learning model can incrementally adapt to the new episodes, {\it without forgetting}…
• Intra-Processing Methods for Debiasing Neural Networks
  15 Jun 2020 4 repositories listed Syntology ran 5 of 5 samples · 0 unverified
  Intra-processing methods are designed specifically to debias large models which have been trained on a generic dataset and fine-tuned on a more specific task.
• Genetic programming approaches to learning fair classifiers
  28 Apr 2020 4 repositories listed
  In this paper, current approaches to fairness are discussed and used to motivate algorithmic proposals that incorporate fairness into genetic programming for classification.
• Fair Regression: Quantitative Definitions and Reduction-based Algorithms
  30 May 2019 4 repositories listed Syntology ran 2 of 2 samples · 0 unverified
  Our schemes only require access to standard risk minimization algorithms (such as standard classification or least-squares regression) while providing theoretical guarantees on the optimality and fairness of the…
• Classification with Fairness Constraints: A Meta-Algorithm with Provable Guarantees
  15 Jun 2018 4 repositories listed
  The main contribution of this paper is a new meta-algorithm for classification that takes as input a large class of fairness constraints, with respect to multiple non-disjoint sensitive attributes, and which comes with…
• A comparative study of fairness-enhancing interventions in machine learning
  13 Feb 2018 4 repositories listed Syntology ran 10 of 11 samples · 1 unverified · 7 pointer-only (licence)
  Concretely, we present the results of an open benchmark we have developed that lets us compare a number of different algorithms under a variety of fairness measures, and a large number of existing datasets.
• Mitigating Unwanted Biases with Adversarial Learning
  22 Jan 2018 4 repositories listed
  Machine learning is a tool for building models that accurately represent input training data.
• FA*IR: A Fair Top-k Ranking Algorithm
  20 Jun 2017 4 repositories listed Syntology ran 1 of 1 samples · 0 unverified · 1 pointer-only (licence)
  In this work, we define and solve the Fair Top-k Ranking problem, in which we want to determine a subset of k candidates from a large pool of n >> k candidates, maximizing utility (i.
• A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges
  4 Jan 2025 3 repositories listed
  Multimodal Vision Language Models (VLMs) have emerged as a transformative topic at the intersection of computer vision and natural language processing, enabling machines to perceive and reason about the world through…
• Fairer Analysis and Demographically Balanced Face Generation for Fairer Face Verification
  4 Dec 2024 3 repositories listed
  Face recognition and verification are two computer vision tasks whose performances have advanced with the introduction of deep representations.
• When is Multicalibration Post-Processing Necessary?
  10 Jun 2024 3 repositories listed Syntology ran 8 of 8 samples · 0 unverified
  Calibration is a well-studied property of predictors which guarantees meaningful uncertainty estimates.

Syntology lines on 20 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.