Papers › MV-MR: multi-views and multi-representations for self-supervised learning and...
MV-MR: multi-views and multi-representations for self-supervised learning and knowledge distillation
Vitaliy Kinakh, Mariia Drozdova, Slava Voloshynovskiy
We present a new method of self-supervised learning and knowledge distillation based on the multi-views and multi-representations (MV-MR). The MV-MR is based on the maximization of dependence between learnable embeddings from augmented and non-augmented views, jointly with the maximization of dependence between learnable embeddings from augmented view and multiple non-learnable representations from non-augmented view. We show that the proposed method can be used for efficient self-supervised classification and model-agnostic knowledge distillation. Unlike other self-supervised techniques, our approach does not use any contrastive learning, clustering, or stop gradients. MV-MR is a generic framework allowing the incorporation of constraints on the learnable embeddings via the usage of image multi-representations as regularizers. Along this line, knowledge distillation is considered a particular case of such a regularization. MV-MR provides the state-of-the-art performance on the STL10 and ImageNet-1K datasets among non-contrastive and clustering-free methods. We show that a lower complexity ResNet50 model pretrained using proposed knowledge distillation based on the CLIP ViT model achieves state-of-the-art performance on STL10 linear evaluation. The code is available at: https://github.com/vkinakh/mv-mr
Code
Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.
Code Syntology ran Syntology
Not run by Syntology. Nothing on this page verifies that the listed code works.
Tasks
Results from the paper archive 2025-07-28
| Task | Dataset | Model | Metric | Value | Rank at snapshot | Leaderboard | Report |
|---|---|---|---|---|---|---|---|
| Knowledge Distillation | CIFAR-100 | MV-MR (T: CLIP/ViT-B-16 S: resnet50) | Top-1 Accuracy (%) | 78.6 | #3 of 27 | Archive leaderboard | report |
| Self-Supervised Image Classification | ImageNet | MV-MR | Top 1 Accuracy | 74.5% | #83 of 144 | Archive leaderboard | report |
| Self-Supervised Image Classification | ImageNet | MV-MR | Top 5 Accuracy | 92.1 | #83 of 144 | Archive leaderboard | report |
| Self-Supervised Learning | STL-10 | MV-MR | Accuracy | 89.67 | #3 of 3 | Archive leaderboard | report |
| Unsupervised Image Classification | CIFAR-20 | MV-MR | Accuracy | 73.2 | #1 of 14 | Archive leaderboard | report |
| Unsupervised Image Classification | STL-10 | MV-MR | Accuracy | 89.67 | #3 of 9 | Archive leaderboard | report |
Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.
Methods
Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections