Browse State-of-the-Art › Mutual Information Estimation
Mutual Information Estimation
43 papers with code · 0 benchmarks · 0 datasets archive 2025-07-28
To estimate mutual information from samples, specially for high-dimensional variables.
Description from the archive archive 2025-07-28.
Benchmarks archive 2025-07-28
No benchmark for this task in the archive.
Libraries
Not in the archive: the export carries no per-task library table, so there is nothing to show at snapshot 2025-07-28.
Datasets archive 2025-07-28
No dataset record in the archive lists this task.
Subtasks archive 2025-07-28
No subtask under this task in the archive's task tree.
Papers archive 2025-07-28
archive papers tagged: 91 · with a code link: 43 · where Syntology ran a sample: 9 (7 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (9 of 91 tagged: 7 with a run with no instrument failure, 2 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 91 of 91, in archive order: by repositories listed in the archive (most first), then newest first, not by stars (the archive holds no stars, so PwC's “Social” and “Latest” sorts cannot be reproduced). Papers that list no repository come after every paper that lists one.
Papers without a page here are shown as plain text. A Syntology line reads “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the figure “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code); hover it for the repositories the samples that ran came from. The first sentence of the abstract is shown for the first 30 papers; every abstract is on its paper's page.
-
20 Aug 2018 8 repositories listed Syntology community repositories only · 5 ran (of which 2 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 1 unverified (of 6 harvested samples) · 2 pointer-only (licence)In this work, we perform unsupervised learning of representations by maximizing mutual information between an input and the output of a deep neural network encoder.
-
30 Jan 2022 2 repositories listedGraph neural networks (GNNs) have become a powerful tool for processing graph-structured data but still face challenges in effectively aggregating and propagating information between layers, which limits their…
-
22 Nov 2021 2 repositories listedWe then estimate the instantaneous rate of change of the bridge distributions indexed by time (the "time score") -- a quantity defined analogously to data (Stein) scores -- with a novel time score matching objective.
-
13 May 2025 1 repository listedWe employ a unified framework for computing the information capacity of biological signaling systems using Fisher Information.
-
29 Feb 2024 1 repository listed Syntology official (archive's flag): 6 ran · 6 ran (of which 0 constructed an object rather than computing a result; 3 with no instrument failure: 0 honoured, 0 violated, 3 with no contract checked; 3 where Syntology's instrument failed) · 1 unverified (of 7 harvested samples)This research provides an initial exploration of trustworthiness modeling during LLM pre-training, seeking to unveil new insights and spur further developments in the field.
-
19 Jan 2024 1 repository listed Syntology official (archive's flag): 1 ran · 1 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 0 where Syntology's instrument failed) · 0 unverified (of 1 harvested sample)To this end, we propose to extract a language-space noise embedding from the N-best list to represent the noise conditions of source speech, which can promote the denoising process in GER.
-
16 Oct 2023 1 repository listedThe pointwise mutual information profile, or simply profile, is the distribution of pointwise mutual information for a given pair of random variables.
-
13 Oct 2023 1 repository listed Syntology official (archive's flag): 15 ran · 15 ran (of which 0 constructed an object rather than computing a result; 14 with no instrument failure: 2 honoured, 2 violated, 10 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 17 harvested samples) · 17 pointer-only (licence)In this work we present a new method for the estimation of Mutual Information (MI) between random variables.
-
12 Oct 2023 1 repository listedIn our proposed loss function, we apply three methods to estimate the weights of negative pairs and show that the vanilla InfoNCE loss is a special case of Soft-InfoNCE.
-
7 Aug 2023 1 repository listedSpecifically, 1) we maximize the MI between visual and semantic space for distribution alignment; 2) we leverage the temporal information for estimating the MI by encouraging MI to increase as more frames are observed.
-
31 May 2023 1 repository listedWe also see that there is an increase in DSMI with the class label over time.
-
31 May 2023 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 3 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)In this paper, we propose a novel class of discriminative mutual information estimators based on the variational representation of the f-divergence.
-
19 May 2023 1 repository listedComputing optimal transport (OT) for general high-dimensional data has been a long-standing challenge.
-
13 Mar 2023 1 repository listed Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 2 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 4 unverified (of 7 harvested samples) · 7 pointer-only (licence)Since accurate estimation of MI without density information requires a sample size exponential in the true MI, we assume either a single marginal or the full joint density information is known.
-
21 Feb 2023 1 repository listedThe rise of mobile AI accelerators allows latency-sensitive applications to execute lightweight Deep Neural Networks (DNNs) on the client side.
-
28 Jan 2023 1 repository listedWasserstein Discrepancy Measurement builds upon the 2-Wasserstein distance, which is more robust, more efficient in small batch sizes, and able to model the uncertainty of stochastic augmentation processes.
-
19 Jan 2023 1 repository listedWe introduce an information-theoretic quantity with similar properties to mutual information that can be estimated from data without making explicit assumptions on the underlying distribution.
-
25 Nov 2022 1 repository listed Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 0 honoured, 0 violated, 5 with no contract checked; 0 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples)Probability density estimation from observed data constitutes a central task in statistics.
-
15 Oct 2022 1 repository listediGCL designs the invariant-discriminative loss (ID loss) to learn invariant and discriminative representations.
-
25 Jul 2022 1 repository listedTo alleviate this negative effect, in this paper, we investigate the dependence between outputs of the target model and input adversarial samples from the perspective of information theory, and propose an adversarial…
-
20 Apr 2022 1 repository listedThis paper introduces UDA-VAE++, an unsupervised domain adaptation framework for cardiac segmentation with a compact loss function lower bound.
-
5 Jul 2021 1 repository listedDensity ratio estimation serves as an important technique in the unsupervised machine learning toolbox.
-
2 Jul 2021 1 repository listedSuccessful applications of InfoNCE and its variants have popularized the use of contrastive variational mutual information (MI) estimators in machine learning.
-
8 Mar 2021 1 repository listedWe propose and demonstrate a representation learning approach by maximizing the mutual information between local features of images and text.
-
25 Feb 2021 1 repository listedWe propose a novel estimator of the mutual information between two ordinal vectors x and y.
-
21 Dec 2020 1 repository listedEstimating the entropy of a discrete random variable is a fundamental problem in information theory and related fields.
-
23 Nov 2020 1 repository listedExperimental results show that our proposed method achieves significant performance improvements over the state-of-the-art pretrained cross-lingual language model in the CLCD setting.
-
17 Nov 2020 1 repository listedWe argue that the high variance characteristic is due to the uncontrolled complexity of the critic's hypothesis space.
-
22 Jun 2020 1 repository listed Syntology 1 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 3 harvested samples)Density-ratio estimation via classification is a cornerstone of unsupervised learning.
-
27 May 2020 1 repository listedCopula Entropy is a mathematical concept defined by Ma and Sun for multivariate statistical independence measuring and testing, and also proved to be closely related to conditional independence (or transfer entropy).
-
15 May 2020 1 repository listed
-
4 Feb 2020 1 repository listed
-
5 Sep 2019 1 repository listed
-
5 Jun 2019 1 repository listed
-
30 May 2019 1 repository listed
-
28 May 2019 1 repository listed
-
27 May 2019 1 repository listed
-
7 Mar 2019 1 repository listed
-
11 Oct 2018 1 repository listed
-
1 Jul 2018 1 repository listed
-
27 Jan 2018 1 repository listed
-
19 Sep 2017 1 repository listed
-
Distilling Transitional Pattern to Large Language Models for Multimodal Session-based Recommendation13 Apr 2025 0 repositories listed
-
A Deep Bayesian Nonparametric Framework for Robust Mutual Information Estimation11 Mar 2025 0 repositories listed
-
Meta-Statistical Learning: Supervised Learning of Statistical Inference17 Feb 2025 0 repositories listed
-
InfoBridge: Mutual Information estimation via Bridge Matching3 Feb 2025 0 repositories listed
-
The Decoupling Concept Bottleneck Model1 Nov 2024 0 repositories listed
-
fastHDMI: Fast Mutual Information Estimation for High-Dimensional Data14 Oct 2024 0 repositories listed
-
Disentangled Noisy Correspondence Learning10 Aug 2024 0 repositories listed
-
Artifact Feature Purification for Cross-domain Detection of AI-generated Images17 Mar 2024 0 repositories listed
-
Mutual Information Estimation via Normalizing Flows4 Mar 2024 0 repositories listed
-
On regularized Radon-Nikodym differentiation15 Aug 2023 0 repositories listed
-
Subgraph Networks Based Contrastive Learning6 Jun 2023 0 repositories listed
-
On the Effectiveness of Hybrid Mutual Information Estimation1 Jun 2023 0 repositories listed
-
Estimating the Density Ratio between Distributions with High Discrepancy using Multinomial Logistic Regression1 May 2023 0 repositories listed
-
Representation Learning with Information Theory for COVID-19 Detection4 Jul 2022 0 repositories listed
-
Mutual information estimation for graph convolutional neural networks31 Mar 2022 0 repositories listed
-
A Perspective on Neural Capacity Estimation: Viability and Reliability22 Mar 2022 0 repositories listed
-
Neural Topic Modeling with Deep Mutual Information Estimation12 Mar 2022 0 repositories listed
-
CNN-Aided Factor Graphs with Estimated Mutual Information Features for Seizure Detection11 Mar 2022 0 repositories listed
-
Compressed Predictive Information Coding3 Mar 2022 0 repositories listed
-
Toward Enhanced Robustness in Unsupervised Graph Representation Learning: A Graph Information Bottleneck Perspective21 Jan 2022 0 repositories listed
-
A Reverse Jensen Inequality Result with Application to Mutual Information Estimation12 Nov 2021 0 repositories listed
-
Data-Driven Representations for Testing Independence: Modeling, Analysis and Connection with Mutual Information Estimation27 Oct 2021 0 repositories listed
-
Mutual Information Estimation as a Difference of Entropies for Unsupervised Representation Learning29 Sep 2021 0 repositories listed
-
Learning Bias-Invariant Representation by Cross-Sample Mutual Information Minimization11 Aug 2021 0 repositories listed
-
Causal Order Identification to Address Confounding: Binary Variables10 Aug 2021 0 repositories listed
-
Fair Representation Learning using Interpolation Enabled Disentanglement31 Jul 2021 0 repositories listed
-
Decomposed Mutual Information Estimation for Contrastive Representation Learning25 Jun 2021 0 repositories listed
-
Novelty Detection via Contrastive Learning with Negative Data Augmentation18 Jun 2021 0 repositories listed
-
Frustratingly Easy Transferability Estimation17 Jun 2021 0 repositories listed
-
Reinforce Security: A Model-Free Approach Towards Secure Wiretap Coding1 Jun 2021 0 repositories listed
-
Rethinking InfoNCE: How Many Negative Samples Do You Need?27 May 2021 0 repositories listed
-
A Quantitative Metric for Privacy Leakage in Federated Learning24 Feb 2021 0 repositories listed
-
Multimodal Image-to-Image Translation via Mutual Information Estimation and Maximization8 Aug 2020 0 repositories listed
-
Multi-label Contrastive Predictive Coding20 Jul 2020 0 repositories listed
-
Alleviating the Incompatibility between Cross Entropy Loss and Episode Training for Few-shot Skin Disease Classification21 Apr 2020 0 repositories listed
-
On Information Plane Analyses of Neural Network Classifiers -- A Review21 Mar 2020 0 repositories listed
-
Matching Text with Deep Mutual Information Estimation9 Mar 2020 0 repositories listed
-
Exploiting Style and Attention in Real-World Super-Resolution21 Dec 2019 0 repositories listed
-
Improved Mutual Information Estimation25 Sep 2019 0 repositories listed
-
Deep Spectral Clustering using Dual Autoencoder Network30 Apr 2019 0 repositories listed
-
Biphasic Learning of GANs for High-Resolution Image-to-Image Translation14 Apr 2019 0 repositories listed
-
Adaptive Estimators Show Information Compression in Deep Neural Networks24 Feb 2019 0 repositories listed
-
Nonparanormal Information Estimation24 Feb 2017 0 repositories listed
-
Mixed vine copulas as joint models of spike counts and local field potentials1 Dec 2016 0 repositories listed
-
Theoretical Evaluation of Feature Selection Methods based on Mutual Information21 Sep 2016 0 repositories listed
-
An Empirical Study of Leading Measures of Dependence9 May 2015 0 repositories listed
-
Equitability Analysis of the Maximal Information Coefficient, with Comparisons27 Jan 2013 0 repositories listed
-
Exponential Concentration for Mutual Information Estimation with Application to Forests1 Dec 2012 0 repositories listed
Syntology lines on 9 of the papers shown; no Syntology record for the others (a paper without an arXiv id cannot be joined to the graph, and absence from the graph layer is not a recorded non-run). “Ran” means the sample executed on a synthesized fixture, not that the paper's result was reproduced; each line links to that paper's sample list. Syntology's record for this page last changed ; when this build read Syntology's graph is in the build record. For agents: get_harvested_code_for_paper(arxiv_id) lists each paper's samples; how to connect.
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections