</> Code with Papers by Syntology

Methods › General › Self-Supervised Learning › ReLIC

ReLIC

12 papers tagged archive 2025-07-28

Introduced by Jovana Mitrovic et al. in Representation Learning via Invariant Causal Mechanisms

archive 2025-07-28 Description, source and code snippet are the archive's method entry.

ReLIC, or Representation Learning via Invariant Causal Mechanisms, is a self-supervised learning objective that enforces invariant prediction of proxy targets across augmentations through an invariance regularizer which yields improved generalization guarantees.

We can write the objective as:

X𝔼 ∼_(l k), a_(q 𝒜)𝔼 ∑_(b ∈a_(l k), a_(q t)) ℒ_b(Yᴿ, f(X)) s.t. K L(p^(d o(a_(l k)))(Yᴿ |f(X)), p^(d o(a_(q t)))(Yᴿ |f(X))) ≤ρ

where ℒ is the proxy task loss and K L is the Kullback-Leibler (KL) divergence. Note that any distance measure on distributions can be used in place of the KL divergence.

Concretely, as proxy task we associate to every datapoint xᵢ the label yᵢᴿ=i. This corresponds to the instance discrimination task, commonly used in contrastive learning. We take pairs of points (xᵢ, xⱼ) to compute similarity scores and use pairs of augmentations a_(l k)=(aₗ, aₖ) ∈ 𝒜 ×𝒜 to perform a style intervention. Given a batch of samples xᵢᵢ₌₁ᴺ ∼𝒟, we use

p^(d o(a_(l k)))(Yᴿ=j |f(xᵢ)) ∝exp(ϕ(f(xᵢ^(aₗ)), h(xⱼ^(aₖ))) / τ)

with xᵃ data augmented with a and τ a softmax temperature parameter. We encode f using a neural network and choose h to be related to f, e.g. h=f or as a network with an exponential moving average of the weights of f (e.g. target networks). To compare representations we use the function ϕ(f(xᵢ), h(xⱼ))=⟨g(f(xᵢ)), g(h(xⱼ))⟩ where g is a fully-connected neural network often called the critic.

Combining these pieces, we learn representations by minimizing the following objective over the full set of data xᵢ ∈𝒟 and augmentations a_(l k) ∈𝒜 ×𝒜

-∑ᵢ₌₁ᴺ ∑_(a_(l k)) log(exp(ϕ(f(xᵢ^(aₗ)), h(xᵢ^(aₖ))) / τ))/(∑ₘ₌₁ᴹ exp(ϕ(f(xᵢ^(aₗ)), h(xₘ^(aₖ))) / τ))+α∑_(a_(l k), a_(q t)) K L(p^(d o(a_(l k))), p^(d o(a_(q t))))

with M the number of points we use to construct the contrast set and α the weighting of the invariance penalty. The shorthand p^(d o(a)) is used for p^(d o(a))(Yᴿ=j |f(xᵢ)). The Figure shows a schematic of the RELIC objective.

PaperSource

Papers archive 2025-07-28

12 shown of 12, newest first. Repository counts are the archive's code-links table. A Syntology line states what Syntology ran from that paper's harvested code; it is per sample and not a correctness claim.

Tasks archive 2025-07-28

20 shown of 23 tasks the archive attaches to papers tagged with this method, by distinct papers. A task without a page in the catalog is plain text.

TaskPapers
Self-Supervised Learning3
Contrastive Learning2
Image Classification2
Reinforcement Learning (RL)2
Representation Learning2
Self-Supervised Image Classification2
Semantic Segmentation2
image-classification2
Few-Shot Imitation Learning1
Imitation Learning1
In-Context Learning1
In-Context Reinforcement Learning1
Instruction Following1
Linear evaluation1
Object Detection1
Out-of-Distribution Generalization1
Reinforcement Learning1
Semi-Supervised Image Classification1
Symbolic Regression1
model1

Usage over time archive 2025-07-28

Papers per year tagged with ReLIC: 2020 to 2025, peak 5 5 0 2020: 1 paper 2020 2021: 1 paper 2021 2022: 3 papers 2022 2023: 0 papers 2023 2024: 5 papers 2024 2025: 2 papers 2025
Papers per year the archive tags with this method, by the paper's archive date (12 dated). Bars are counts, not a trend claim.

Components: the archive holds no method-to-method composition, so PwC's Components table cannot be rebuilt; the Papers list carries no Results column for the same reason (the archive does not join its leaderboard rows to method tags).

Categories archive 2025-07-28

Self-Supervised Learning

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections