Mutual Information Estimation

Description from the archive archive 2025-07-28.

Benchmarks archive 2025-07-28

No benchmark for this task in the archive.

Libraries

Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.

Datasets archive 2025-07-28

No dataset record in the archive lists this task.

Subtasks archive 2025-07-28

No subtask under this task in the archive's task tree.

Most implemented papers archive 2025-07-28

30 shown of 43 papers with code (91 tagged with this task in all), ordered by repositories listed in the archive, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers without a page here are shown as plain text.

• Learning deep representations by mutual information estimation and maximization
  20 Aug 2018 8 repositories listed Syntology ran 5 of 6 samples · 1 unverified · 2 pointer-only (licence)
  In this work, we perform unsupervised learning of representations by maximizing mutual information between an input and the output of a deep neural network encoder.
• Graph Representation Learning via Aggregation Enhancement
  30 Jan 2022 2 repositories listed
  Graph neural networks (GNNs) have become a powerful tool for processing graph-structured data but still face challenges in effectively aggregating and propagating information between layers, which limits their…
• Density Ratio Estimation via Infinitesimal Classification
  22 Nov 2021 2 repositories listed
  We then estimate the instantaneous rate of change of the bridge distributions indexed by time (the "time score") -- a quantity defined analogously to data (Stein) scores -- with a novel time score matching objective.
• Closed-Form Information Capacity of Canonical Signaling Models
  13 May 2025 1 repository listed
  We employ a unified framework for computing the information capacity of biological signaling systems using Fisher Information.
• Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Models
  29 Feb 2024 1 repository listed Syntology ran 6 of 7 samples · 1 unverified
  This research provides an initial exploration of trustworthiness modeling during LLM pre-training, seeking to unveil new insights and spur further developments in the field.
• Large Language Models are Efficient Learners of Noise-Robust Speech Recognition
  19 Jan 2024 1 repository listed Syntology ran 1 of 1 samples · 0 unverified
  To this end, we propose to extract a language-space noise embedding from the N-best list to represent the noise conditions of source speech, which can promote the denoising process in GER.
• On the Properties and Estimation of Pointwise Mutual Information Profiles
  16 Oct 2023 1 repository listed
  The pointwise mutual information profile, or simply profile, is the distribution of pointwise mutual information for a given pair of random variables.
• MINDE: Mutual Information Neural Diffusion Estimation
  13 Oct 2023 1 repository listed Syntology ran 15 of 17 samples · 2 unverified · 17 pointer-only (licence)
  In this work we present a new method for the estimation of Mutual Information (MI) between random variables.
• Rethinking Negative Pairs in Code Search
  12 Oct 2023 1 repository listed
  In our proposed loss function, we apply three methods to estimate the weights of negative pairs and show that the vanilla InfoNCE loss is a special case of Soft-InfoNCE.
• Zero-shot Skeleton-based Action Recognition via Mutual Information Estimation and Maximization
  7 Aug 2023 1 repository listed
  Specifically, 1) we maximize the MI between visual and semantic space for distribution alignment; 2) we leverage the temporal information for estimating the MI by encouraging MI to increase as more frames are observed.
• Mutual Information Estimation via f-Divergence and Data Derangements
  31 May 2023 1 repository listed Syntology ran 3 of 3 samples · 0 unverified
  In this paper, we propose a novel class of discriminative mutual information estimators based on the variational representation of the f-divergence.
• Assessing Neural Network Representations During Training Using Data Diffusion Spectra
  31 May 2023 1 repository listed
  We also see that there is an increase in DSMI with the class label over time.
• Computing high-dimensional optimal transport by flow neural networks
  19 May 2023 1 repository listed
  Computing optimal transport (OT) for general high-dimensional data has been a long-standing challenge.
• Improving Mutual Information Estimation with Annealed and Energy-Based Bounds
  13 Mar 2023 1 repository listed Syntology ran 3 of 7 samples · 4 unverified · 7 pointer-only (licence)
  Since accurate estimation of MI without density information requires a sample size exponential in the true MI, we assume either a single marginal or the full joint density information is known.
• FrankenSplit: Efficient Neural Feature Compression with Shallow Variational Bottleneck Injection for Mobile Edge Computing
  21 Feb 2023 1 repository listed
  The rise of mobile AI accelerators allows latency-sensitive applications to execute lightweight Deep Neural Networks (DNNs) on the client side.
• Mutual Wasserstein Discrepancy Minimization for Sequential Recommendation
  28 Jan 2023 1 repository listed
  Wasserstein Discrepancy Measurement builds upon the 2-Wasserstein distance, which is more robust, more efficient in small batch sizes, and able to model the uncertainty of stochastic augmentation processes.
• DiME: Maximizing Mutual Information by a Difference of Matrix-Based Entropies
  19 Jan 2023 1 repository listed
  We introduce an information-theoretic quantity with similar properties to mutual information that can be estimated from data without making explicit assumptions on the underlying distribution.
• Copula Density Neural Estimation
  25 Nov 2022 1 repository listed Syntology ran 0 of 7 samples · 7 unverified
  Probability density estimation from observed data constitutes a central task in statistics.
• Augmentation-Free Graph Contrastive Learning of Invariant-Discriminative Representations
  15 Oct 2022 1 repository listed
  iGCL designs the invariant-discriminative loss (ID loss) to learn invariant and discriminative representations.
• Improving Adversarial Robustness via Mutual Information Estimation
  25 Jul 2022 1 repository listed
  To alleviate this negative effect, in this paper, we investigate the dependence between outputs of the target model and input adversarial samples from the perspective of information theory, and propose an adversarial…
• Unsupervised Domain Adaptation for Cardiac Segmentation: Towards Structure Mutual Information Maximization
  20 Apr 2022 1 repository listed
  This paper introduces UDA-VAE++, an unsupervised domain adaptation framework for cardiac segmentation with a compact loss function lower bound.
• Featurized Density Ratio Estimation
  5 Jul 2021 1 repository listed
  Density ratio estimation serves as an important technique in the unsupervised machine learning toolbox.
• Tight Mutual Information Estimation With Contrastive Fenchel-Legendre Optimization
  2 Jul 2021 1 repository listed
  Successful applications of InfoNCE and its variants have popularized the use of contrastive variational mutual information (MI) estimators in machine learning.
• Multimodal Representation Learning via Maximization of Local Mutual Information
  8 Mar 2021 1 repository listed
  We propose and demonstrate a representation learning approach by maximizing the mutual information between local features of images and text.
• MIND: Inductive Mutual Information Estimation, A Convex Maximum-Entropy Copula Approach
  25 Feb 2021 1 repository listed
  We propose a novel estimator of the mutual information between two ordinal vectors x and y.
• Neural Joint Entropy Estimation
  21 Dec 2020 1 repository listed
  Estimating the entropy of a discrete random variable is a fundamental problem in information theory and related fields.
• Unsupervised Domain Adaptation of a Pretrained Cross-Lingual Language Model
  23 Nov 2020 1 repository listed
  Experimental results show that our proposed method achieves significant performance improvements over the state-of-the-art pretrained cross-lingual language model in the CLCD setting.
• Reducing the Variance of Variational Estimates of Mutual Information by Limiting the Critic's Hypothesis Space to RKHS
  17 Nov 2020 1 repository listed
  We argue that the high variance characteristic is due to the uncontrolled complexity of the critic's hypothesis space.
• Telescoping Density-Ratio Estimation
  22 Jun 2020 1 repository listed Syntology ran 1 of 3 samples · 2 unverified
  Density-ratio estimation via classification is a cornerstone of unsupervised learning.
• copent: Estimating Copula Entropy and Transfer Entropy in R
  27 May 2020 1 repository listed
  Copula Entropy is a mathematical concept defined by Ma and Sun for multivariate statistical independence measuring and testing, and also proved to be closely related to conditional independence (or transfer entropy).

Syntology lines on 8 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced. Read from the graph 2026-09-24.