Methods › General › Self-Supervised Learning › ReLIC
ReLIC
Introduced by Jovana Mitrovic et al. in Representation Learning via Invariant Causal Mechanisms
archive 2025-07-28 Description, source and code snippet are the archive's method entry.
ReLIC, or Representation Learning via Invariant Causal Mechanisms, is a self-supervised learning objective that enforces invariant prediction of proxy targets across augmentations through an invariance regularizer which yields improved generalization guarantees.
We can write the objective as:
X𝔼 ∼_(l k), a_(q 𝒜)𝔼 ∑_(b ∈a_(l k), a_(q t)) ℒ_b(Yᴿ, f(X)) s.t. K L(p^(d o(a_(l k)))(Yᴿ |f(X)), p^(d o(a_(q t)))(Yᴿ |f(X))) ≤ρ
where ℒ is the proxy task loss and K L is the Kullback-Leibler (KL) divergence. Note that any distance measure on distributions can be used in place of the KL divergence.
Concretely, as proxy task we associate to every datapoint xᵢ the label yᵢᴿ=i. This corresponds to the instance discrimination task, commonly used in contrastive learning. We take pairs of points (xᵢ, xⱼ) to compute similarity scores and use pairs of augmentations a_(l k)=(aₗ, aₖ) ∈ 𝒜 ×𝒜 to perform a style intervention. Given a batch of samples xᵢᵢ₌₁ᴺ ∼𝒟, we use
p^(d o(a_(l k)))(Yᴿ=j |f(xᵢ)) ∝exp(ϕ(f(xᵢ^(aₗ)), h(xⱼ^(aₖ))) / τ)
with xᵃ data augmented with a and τ a softmax temperature parameter. We encode f using a neural network and choose h to be related to f, e.g. h=f or as a network with an exponential moving average of the weights of f (e.g. target networks). To compare representations we use the function ϕ(f(xᵢ), h(xⱼ))=⟨g(f(xᵢ)), g(h(xⱼ))⟩ where g is a fully-connected neural network often called the critic.
Combining these pieces, we learn representations by minimizing the following objective over the full set of data xᵢ ∈𝒟 and augmentations a_(l k) ∈𝒜 ×𝒜
-∑ᵢ₌₁ᴺ ∑_(a_(l k)) log(exp(ϕ(f(xᵢ^(aₗ)), h(xᵢ^(aₖ))) / τ))/(∑ₘ₌₁ᴹ exp(ϕ(f(xᵢ^(aₗ)), h(xₘ^(aₖ))) / τ))+α∑_(a_(l k), a_(q t)) K L(p^(d o(a_(l k))), p^(d o(a_(q t))))
with M the number of points we use to construct the contrast set and α the weighting of the invariance penalty. The shorthand p^(d o(a)) is used for p^(d o(a))(Yᴿ=j |f(xᵢ)). The Figure shows a schematic of the RELIC objective.
In Syntology about another paper, arXiv:2609.32965: no code under this name (no_code; paper_only). The lookup matched 'Relic' ignoring case. About another paper: this answer is about arXiv:2609.32965, while the archive credits arXiv:2010.07922, so it does not say whether this method has code.
From the lookup's note on this name: The same name also belongs to the paper “RELIC: Investigating Large Language Model Responses using Self-Consistency” (arXiv:2311.16842, 60 citations), which the lookup finds cited far more than the paper this answer is about (citations not recorded), so this answer may be about a different sense of the name. For arXiv:2311.16842: ask the lookup about it.
Syntology's have check for this name (free, it answers coverage only), read 2026-10-05; ask it again live. A level is the served verification level of that one answer; nothing here is summed across methods.
Papers archive 2025-07-28
archive papers tagged: 12 · with a code link: 5 · where Syntology ran a sample: 4 (3 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument) Syntology
Show: all tagged papersonly where code ran (4 of 12 tagged: 3 with a run with no instrument failure, 1 where every run was a failure of Syntology's instrument)
Page 1 of 1: papers 1 to 12 of 12, newest first by the archive's date (ties by slug), in archive order.
Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code, as “N ran (of which C constructed an object rather than computing a result; K with no instrument failure: H honoured, V violated, P with no contract checked; I where Syntology's instrument failed) · U unverified”; the instrument figure counts failures of Syntology's instrument, not of the code. It is per sample and not a correctness claim. When the archive marks a repository official for the paper, the line starts with that repository's state (the archive's flag, not a verdict on who wrote the code; “community repositories only” when every sample that ran came from a community repository, “official: no sample here; runs from other or unrecorded repositories” when some came from a repository the paper names or has in its text, or from none recorded); hover it for the repositories the samples that ran came from.
-
RELIC: Evaluating Compositional Instruction Following via Language Recognition 5 Jun 2025 · 0 repositories · arXiv:2506.05205
-
A Collaborative Jade Recognition System for Mobile Devices Based on Lightweight and Large Models 20 Feb 2025 · 0 repositories · arXiv:2502.14332
-
An RNA condensate model for the origin of life 6 Dec 2024 · 0 repositories · arXiv:2412.05396
-
MRIFE: A Mask-Recovering and Interactive-Feature-Enhancing Semantic Segmentation Network For Relic Landslide Detection 26 Nov 2024 · 0 repositories · arXiv:2411.17167
-
Machine-Learning Analysis of Radiative Decays to Dark Matter at the LHC 17 Oct 2024 · 0 repositories · arXiv:2410.13799
-
ReLIC: A Recipe for 64k Steps of In-Context Reinforcement Learning for Embodied AI 3 Oct 2024 · 1 repository · arXiv:2410.02751Syntology official (archive's flag): 5 ran · 5 ran (of which 0 constructed an object rather than computing a result; 4 with no instrument failure: 0 honoured, 0 violated, 4 with no contract checked; 1 where Syntology's instrument failed) · 2 unverified (of 7 harvested samples) · 7 pointer-only (licence)
-
Symbolic Regression for Beyond the Standard Model Physics 28 May 2024 · 0 repositories · arXiv:2405.18471
-
Self-Supervised Learning Through Efference Copies 17 Oct 2022 · 1 repository · arXiv:2210.09224
-
Mixed Anhydrides at the Intersection Between Peptide and RNA Autocatalytic Sets: Evolution of Biological Coding 25 May 2022 · 0 repositories · arXiv:2208.01491
-
Pushing the limits of self-supervised ResNets: Can we outperform supervised learning without labels on ImageNet? 13 Jan 2022 · 1 repository · arXiv:2201.05119Syntology official (archive's flag): 13 ran · 13 ran (of which 0 constructed an object rather than computing a result; 13 with no instrument failure: 0 honoured, 0 violated, 13 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified (of 14 harvested samples)
-
CoBERL: Contrastive BERT for Reinforcement Learning 12 Jul 2021 · 2 repositories · arXiv:2107.05431Syntology official (archive's flag): 3 ran · 3 ran (of which 0 constructed an object rather than computing a result; 1 with no instrument failure: 0 honoured, 1 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 3 harvested samples)
-
Representation Learning via Invariant Causal Mechanisms 15 Oct 2020 · 2 repositories · arXiv:2010.07922Syntology 2 ran (of which 0 constructed an object rather than computing a result; 0 with no instrument failure: 0 honoured, 0 violated, 0 with no contract checked; 2 where Syntology's instrument failed) · 0 unverified (of 2 harvested samples) · 1 pointer-only (licence)
Tasks archive 2025-07-28
20 shown of 23 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.
Usage over time archive 2025-07-28
Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).
Categories archive 2025-07-28
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections