Papers › Bootstrap your own latent: A new approach to self-supervised Learning

Bootstrap your own latent: A new approach to self-supervised Learning

13 Jun 2020arXiv:2006.07733archive 2025-07-28

Jean-bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, Michal Valko

We introduce Bootstrap Your Own Latent (BYOL), a new approach to self-supervised image representation learning. BYOL relies on two neural networks, referred to as online and target networks, that interact and learn from each other. From an augmented view of an image, we train the online network to predict the target network representation of the same image under a different augmented view. At the same time, we update the target network with a slow-moving average of the online network. While state-of-the art methods rely on negative pairs, BYOL achieves a new state of the art without them. BYOL reaches 74.3% top-1 classification accuracy on ImageNet using a linear evaluation with a ResNet-50 architecture and 79.6% with a larger ResNet. We show that BYOL performs on par or better than the current state of the art on both transfer and semi-supervised benchmarks. Our implementation and pretrained models are given on GitHub.

PaperPDFCodeCode Syntology ran

In Syntology Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, Syntology's MCP tool lists every function and class Syntology harvested from this paper and whether it ran (how to connect): get_harvested_code_for_paper(arxiv_id="2006.07733")

Code

Syntology Ran 62 of 79 code samples harvested from 16 repositories linked to this paper; 17 have no recorded run. Of those that ran: 4 ran · honoured contract; 1 ran · violated contract; 19 ran · our draft was wrong; 1 ran · fixture could not drive it; 37 ran with no contract checked.

By repository: community (archive-listed): 75 samples from 16 repositories, 58 ran; 4 identical to code first harvested elsewhere. The run record, sample by sample. “Ran” means executed on a synthesized input, not that the code is correct or reproduces the paper.

31 repositories listed; official and paper-mentioned ones first.

EchoItLiu/SelfGait mentioned on GitHubpytorch report
HibikiJie/BYOL mentioned on GitHubpytorch report
Kennethborup/BYOL mentioned on GitHubpytorch report
Namkyeong/BGRL_Pytorch mentioned on GitHubpytorch report
ReshinthAdith/BYOL-Pytorch mentioned on GitHubpytorchMIT report
SaeedShurrab/SimSiam-pytorch mentioned on GitHubpytorch report
SaeedShurrab/Simple-BYOL mentioned on GitHubpytorch report
Westlake-AI/openmixup mentioned on GitHubpytorch report
facebookresearch/clip-rocket mentioned on GitHubpytorch report
fmi-basel/implicit-var-reg mentioned on GitHubpytorch report
htdt/self-supervised mentioned on GitHubpytorch report
jramapuram/BYOL mentioned on GitHubpytorch report
juneweng/byol-pytorch mentioned on GitHubpytorch report
liyi01827/noisy-contrastive mentioned on GitHubpytorch report
lucidrains/byol-pytorch mentioned on GitHubpytorch report
miszkur/SelfSupervisedLearning mentioned on GitHubtf report
philippmwirth/byol mentioned on GitHubpytorch report
reshinthadithyan/BYOL-Pytorch mentioned on GitHubpytorch report
sabadijou/byol_multi_gpu mentioned on GitHubpytorch report
sthalles/PyTorch-BYOL mentioned on GitHubpytorch report
talipucar/PyFlow_BYOL mentioned on GitHubpytorch report
vturrisi/solo-learn mentioned on GitHubpytorch report
yaox12/BYOL-PyTorch mentioned on GitHubpytorch report
kakaobrain/scrl pytorchApache-2.0 report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

Code Syntology ran Syntology

79 samples harvested; 62 ran; 4 honoured the contract we drafted; 17 have no recorded run. Read from Syntology's graph 2026-09-24; that is when this build read the record, not when the samples ran.

4ran · honoured contract
1ran · violated contract
19ran · our draft was wrong
1ran · fixture could not drive it
37ran
17unverified

Licence: 46 of the 79 samples are pointer only, meaning Syntology does not serve that copy's text. This page shows no code text for any sample; each one links to its file in the repository.

Harvested from 16 repositories linked to this paper, official or community; each sample names its own and says which. Some samples are identical code Syntology first harvested from another repository; for those, this paper's copy is not located and its licence is not recorded. “Ran” means the sample executed on a synthesized input. It does not mean the output is correct, and nothing here reproduces the paper's results. “Honoured” and “violated” refer to a contract Syntology drafted from the code itself; “our draft was wrong” and “fixture could not drive it” are failures of Syntology's instrument, not of the code.

Each sample ends with its code_sha256, Syntology's identity for that exact code. An agent fetches the stored sample with Syntology's MCP tool get_code(code_sha256="…") (how to connect); click an identity to copy that call.

Repository labels, per sample. official repository: The archive marks this repository official for the paper. named in the paper: The archive records that the paper mentions this repository; it is not marked official. community (archive-listed): In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper. found in paper text by Syntology: Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted. community: Not in the archive's code links for this paper; a community repository Syntology harvested. Samples from a repository marked official are listed first. Licence labels name the repository's licence as recorded at harvest. “Pointer only” means Syntology does not serve that copy's text, for one of four reasons: no licence file was found; the licence was not identified; the licence is recorded as permissive but that copy's record is not marked cleared; or the licence is outside the permissive list Syntology serves text under (MIT, Apache-2.0, BSD and similar). Some licences outside that list permit redistribution, such as WTFPL, and GPL-3.0 under its conditions; they are simply not on the list. Hover a licence label for the reason. File links open the file on GitHub at the default branch, which may have changed since the harvest.

BYOLModel yaox12/BYOL-PyTorch/model/byol_model.py community (archive-listed) ran fingerprinted MIT (permissive) · 2784d037b1add7b4 · report
BatchNorm1d liyi01827/noisy-contrastive/simsiam/model_factory.py community (archive-listed) ran · metamorphic tier: deterministic fingerprinted no licence file found · pointer only · 17ab24f7892fa961 · report
CNNEncoder talipucar/PyFlow_BYOL/utils/model_utils.py community (archive-listed) ran · metamorphic tier: invariant fingerprinted no licence file found · pointer only · 9942ab84cb1d15a2 · report
ConvMLP sabadijou/byol_multi_gpu/models/byol.py community (archive-listed) ran · metamorphic tier: invariant fingerprinted no licence file found · pointer only · 096cfb3b42210f32 · report
EMA lucidrains/byol-pytorch/byol_pytorch/byol_pytorch.py community (archive-listed) ran MIT (permissive) · f6d2ad22f5a5580f · report
EMA Kennethborup/BYOL/BYOL/byol.py community (archive-listed) ran fingerprinted MIT (permissive) · 1ef88af7143047a0 · report
EncodeProject talipucar/PyFlow_BYOL/utils/model_utils.py community (archive-listed) ran fingerprinted no licence file found · pointer only · e3cbe47be1db1b63 · report
EncoderwithProjection yaox12/BYOL-PyTorch/model/byol_model.py community (archive-listed) ran fingerprinted MIT (permissive) · 7dfac9bef432ab6b · report
HiddenLayers talipucar/PyFlow_BYOL/utils/model_utils.py community (archive-listed) ran fingerprinted no licence file found · pointer only · 67317765520392ae · report
Hook Kennethborup/BYOL/BYOL/byol.py community (archive-listed) ran MIT (permissive) · 688c1c4f58a24171 · report
Linear liyi01827/noisy-contrastive/simsiam/model_factory.py community (archive-listed) ran · metamorphic tier: deterministic no licence file found · pointer only · 4a24b0e23a6445dd · report
MLP lucidrains/byol-pytorch/byol_pytorch/byol_pytorch.py community (archive-listed) ran · our draft was wrong MIT (permissive) · e27680ba34c9822b · report
MLP yaox12/BYOL-PyTorch/model/byol_model.py community (archive-listed) ran MIT (permissive) · f74419add7dfb458 · report
MLP amazon-research/exponential-moving-average-normalization/models/byol.py community (archive-listed) ran fingerprinted no licence file found · pointer only · 100d08f2dcfe7489 · report
MLP juneweng/byol-pytorch/byol_pytorch/byol_pytorch.py community (archive-listed) ran · metamorphic tier: invariant MIT (permissive) · 66df1a1ee476a147 · report
MLP sabadijou/byol_multi_gpu/models/byol.py community (archive-listed) ran · metamorphic tier: invariant fingerprinted no licence file found · pointer only · dcc9aa7b1d6a5de1 · report
MLP Kennethborup/BYOL/BYOL/byol.py community (archive-listed) ran MIT (permissive) · 91caf7decdc1ce5f · report
MLPHead miszkur/SelfSupervisedLearning/models/model.py community (archive-listed) ran no licence file found · pointer only · 5ec3adcd55340bbd · report
MLP_Base reshinthadithyan/BYOL-Pytorch/BYOL.py community (archive-listed) ran · metamorphic tier: deterministic fingerprinted MIT (permissive) · ad27d03b9ab5ef12 · report
MatryoshkaProjector filipbasara0/matryoshka-representation-learning/mrl/mrl.py community (archive-listed) ran · metamorphic tier: deterministic fingerprinted MIT (permissive) · cf8bd65b80370c1e · report
MaybeSyncBatchnorm lucidrains/byol-pytorch/byol_pytorch/byol_pytorch.py community (archive-listed) ran · our draft was wrong MIT (permissive) · e0ac07cc5ef23be4 · report
ModelWrapper Kennethborup/BYOL/BYOL/byol.py community (archive-listed) ran MIT (permissive) · e5edf7e87108eca5 · report
NetWrapper lucidrains/byol-pytorch/byol_pytorch/byol_pytorch.py community (archive-listed) ran · metamorphic tier: deterministic MIT (permissive) · a8cac4c5cd21f6d9 · report
NetWrapper juneweng/byol-pytorch/byol_pytorch/byol_pytorch.py community (archive-listed) ran MIT (permissive) · 4bb4f89baaa7a100 · report
PreActBlock liyi01827/noisy-contrastive/simsiam/model_factory.py community (archive-listed) ran · metamorphic tier: deterministic fingerprinted no licence file found · pointer only · 9cf7c8e34248b087 · report
PreActBottleneck liyi01827/noisy-contrastive/simsiam/model_factory.py community (archive-listed) ran · metamorphic tier: deterministic fingerprinted no licence file found · pointer only · 48c90dc3cead67c6 · report
Predictor yaox12/BYOL-PyTorch/model/byol_model.py community (archive-listed) ran fingerprinted MIT (permissive) · 03e75d5f5306b451 · report
Predictor talipucar/PyFlow_BYOL/utils/model_utils.py community (archive-listed) ran · metamorphic tier: invariant fingerprinted no licence file found · pointer only · 7641e6d697e6dffa · report
Projector talipucar/PyFlow_BYOL/utils/model_utils.py community (archive-listed) ran · metamorphic tier: invariant fingerprinted no licence file found · pointer only · bd8fca31411b4514 · report
RandomApply lucidrains/byol-pytorch/byol_pytorch/byol_pytorch.py community (archive-listed) ran · metamorphic tier: deterministic fingerprinted MIT (permissive) · faf380e1645f84b0 · report
RandomApply juneweng/byol-pytorch/byol_pytorch/byol_pytorch.py community (archive-listed) ran · metamorphic tier: invariant fingerprinted MIT (permissive) · 8d381c72dbc91f6f · report
RandomApply Kennethborup/BYOL/BYOL/byol.py community (archive-listed) ran · metamorphic tier: invariant fingerprinted MIT (permissive) · 062cc33741812100 · report
ReLIC filipbasara0/matryoshka-representation-learning/mrl/mrl.py community (archive-listed) ran · metamorphic tier: deterministic MIT (permissive) · 37caddd65d22b0fe · report
ResNet liyi01827/noisy-contrastive/simsiam/model_factory.py community (archive-listed) ran · metamorphic tier: deterministic fingerprinted no licence file found · pointer only · ba93c0af999b7a85 · report
ResNet101 liyi01827/noisy-contrastive/simsiam/model_factory.py community (archive-listed) ran · our draft was wrong no licence file found · pointer only · 1d47aaa21a3bb862 · report
ResNet152 liyi01827/noisy-contrastive/simsiam/model_factory.py community (archive-listed) ran · our draft was wrong no licence file found · pointer only · f8d0daa856a02a03 · report
ResNet18 liyi01827/noisy-contrastive/simsiam/model_factory.py community (archive-listed) ran · our draft was wrong no licence file found · pointer only · 1c728412af1388f5 · report
ResNet18 miszkur/SelfSupervisedLearning/models/model.py community (archive-listed) ran no licence file found · pointer only · 3df1625e8f0b9699 · report
ResNet34 liyi01827/noisy-contrastive/simsiam/model_factory.py community (archive-listed) ran · our draft was wrong no licence file found · pointer only · a30de131b4f273c8 · report
ResNet50 liyi01827/noisy-contrastive/simsiam/model_factory.py community (archive-listed) ran · our draft was wrong no licence file found · pointer only · d20bc483f371ec97 · report
SEM lucidrains/byol-pytorch/byol_pytorch/byol_pytorch.py community (archive-listed) ran · metamorphic tier: deterministic MIT (permissive) · d86a281b947c32ef · report
SimSiamMLP lucidrains/byol-pytorch/byol_pytorch/byol_pytorch.py community (archive-listed) ran · our draft was wrong MIT (permissive) · 5d81cae15b63b2d0 · report
SkeletonNet reshinthadithyan/BYOL-Pytorch/BYOL.py community (archive-listed) ran · metamorphic tier: deterministic fingerprinted MIT (permissive) · 55863b2dc4aac43d · report
TripletLoss EchoItLiu/SelfGait/trainer.py community (archive-listed) ran fingerprinted no licence file found · pointer only · e693eee1106320c1 · report
TripletSampler EchoItLiu/SelfGait/trainer.py community (archive-listed) ran no licence file found · pointer only · c9ab21522dc83e5e · report
conv3x3 liyi01827/noisy-contrastive/simsiam/model_factory.py community (archive-listed) ran · our draft was wrong no licence file found · pointer only · ec20f22a185cd708 · report
cosine_loss_fn lucidrains/byol-pytorch/byol_pytorch/byol_pytorch.py community (archive-listed) ran · our draft was wrong fingerprinted MIT (permissive) · aaecf667988ce174 · report
eval_knn htdt/self-supervised/methods/byol.py community (archive-listed) ran · our draft was wrong fingerprinted no licence file found · pointer only · 2ed1c82b5ad2ee9f · report
get_data htdt/self-supervised/methods/byol.py community (archive-listed) ran · fixture could not drive it no licence file found · pointer only · ceeec13af255b5bc · report
get_feature_size filipbasara0/matryoshka-representation-learning/mrl/mrl.py community (archive-listed) ran · honoured contract MIT (permissive) · 74d6aa7ba8ab5ca6 · report
get_head htdt/self-supervised/methods/byol.py community (archive-listed) ran · our draft was wrong no licence file found · pointer only · 2d19f4f3143c2ccf · report
get_resnet talipucar/PyFlow_BYOL/utils/model_utils.py community (archive-listed) ran · our draft was wrong no licence file found · pointer only · d35a3394eb6872b1 · report
l2norm lucidrains/byol-pytorch/byol_pytorch/byol_pytorch.py community (archive-listed) ran · our draft was wrong fingerprinted MIT (permissive) · 09b2eed13fab23fa · report
loss_function HibikiJie/BYOL/utiles/loss_function.py community (archive-listed) ran · honoured contract fingerprinted no licence file found · pointer only · d1695fe8fcdca12a · report
loss_function jramapuram/BYOL/objective.py community (archive-listed) ran · our draft was wrong fingerprinted MIT (permissive) · 5c93a9f666df290c · report
norm_mse_loss htdt/self-supervised/methods/byol.py community (archive-listed) ran · honoured contract fingerprinted no licence file found · pointer only · a2d406a0a54489bf · report
partLoss Kennethborup/BYOL/BYOL/byol.py community (archive-listed) ran · our draft was wrong fingerprinted MIT (permissive) · 2fc1e20b19ae6392 · report
regression_loss jramapuram/BYOL/objective.py community (archive-listed) ran · our draft was wrong fingerprinted MIT (permissive) · ef447d6959435039 · report
BYOL htdt/self-supervised/methods/byol.py community (archive-listed) unverified no licence file found · pointer only · 8808a3600e2448ce · report
BYOL reshinthadithyan/BYOL-Pytorch/BYOL.py community (archive-listed) unverified MIT (permissive) · 418af70ca1e55c34 · report
BYOL lucidrains/byol-pytorch/byol_pytorch/byol_pytorch.py community (archive-listed) unverified MIT (permissive) · 92b4ff27a45e0530 · report
BYOL amazon-research/exponential-moving-average-normalization/models/byol.py community (archive-listed) unverified no licence file found · pointer only · c451e53c0c857a96 · report
BYOL juneweng/byol-pytorch/byol_pytorch/byol_pytorch.py community (archive-listed) unverified MIT (permissive) · a163e55a4f350e6b · report
BYOL talipucar/PyFlow_BYOL/utils/model_utils.py community (archive-listed) unverified no licence file found · pointer only · dfa5e6223a2e37f0 · report
BYOL Kennethborup/BYOL/BYOL/byol.py community (archive-listed) unverified MIT (permissive) · c7965edd69188be9 · report
BYOLTrainer sthalles/PyTorch-BYOL/trainer.py community (archive-listed) unverified no licence file found · pointer only · 58f320d8dc72baf6 · report
BYOLTrainer EchoItLiu/SelfGait/trainer.py community (archive-listed) unverified no licence file found · pointer only · 74a470fbd7774b12 · report
BaseMethod htdt/self-supervised/methods/byol.py community (archive-listed) unverified no licence file found · pointer only · 641188dbe3eb2b34 · report
Encoder amazon-research/exponential-moving-average-normalization/models/byol.py community (archive-listed) unverified no licence file found · pointer only · 6df6dfae3b579fc5 · report
NetWrapper sabadijou/byol_multi_gpu/models/byol.py community (archive-listed) unverified no licence file found · pointer only · 1769f75163afd95d · report
SiameseNetwork miszkur/SelfSupervisedLearning/models/model.py community (archive-listed) unverified no licence file found · pointer only · 5650e93704e0d250 · report
SimSiam liyi01827/noisy-contrastive/simsiam/model_factory.py community (archive-listed) unverified no licence file found · pointer only · 7020323e76ae5b99 · report
_create_model_training_folder sthalles/PyTorch-BYOL/trainer.py community (archive-listed) unverified no licence file found · pointer only · 584ff279cd1e8248 · report
eval_sgd htdt/self-supervised/methods/byol.py community (archive-listed) unverified no licence file found · pointer only · 371021e6723e9c29 · report
get_model htdt/self-supervised/methods/byol.py community (archive-listed) unverified no licence file found · pointer only · 1d4261306685f800 · report
default identical code first harvested elsewhere ran · violated contract licence of this copy not recorded · 1c0f269ae9673b08 · report
flatten identical code first harvested elsewhere ran · honoured contract fingerprinted licence of this copy not recorded · 4daa3e876f494d75 · report
get_module_device identical code first harvested elsewhere ran · our draft was wrong licence of this copy not recorded · f8ef5ca2539fcee5 · report
singleton identical code first harvested elsewhere ran · our draft was wrong licence of this copy not recorded · c6bc247ffa36099c · report

Tasks

Image ClassificationLinear evaluationPerson Re-IdentificationRepresentation LearningSelf-Supervised Image ClassificationSelf-Supervised LearningSelf-Supervised Person Re-IdentificationSemi-Supervised Image Classification

Results from the paper archive 2025-07-28

TaskDatasetModelMetricValueRank at snapshotLeaderboardReport
Image Classification Places205 BYOL Top 1 Accuracy 54.0 #12 of 15 Archive leaderboard report
Person Re-Identification SYSU-30k BYOL (self-supervised) Rank-1 12.7 #5 of 10 Archive leaderboard report
Self-Supervised Image Classification ImageNet BYOL (ResNet-200 x2) Number of Params 250M #33 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet BYOL (ResNet-200 x2) Top 1 Accuracy 79.6% #33 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet BYOL (ResNet-200 x2) Top 5 Accuracy 94.8% #33 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet BYOL (ResNet-50 x4) Number of Params 375M #44 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet BYOL (ResNet-50 x4) Top 1 Accuracy 78.6% #44 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet BYOL (ResNet-50 x4) Top 5 Accuracy 94.2% #44 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet BYOL (ResNet-50 x2) Number of Params 94M #53 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet BYOL (ResNet-50 x2) Top 1 Accuracy 77.4% #53 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet BYOL (ResNet-50 x2) Top 5 Accuracy 93.6% #53 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet BYOL (ResNet-50) Number of Params 24M #85 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet BYOL (ResNet-50) Top 1 Accuracy 74.3% #85 of 144 Archive leaderboard report
Self-Supervised Image Classification ImageNet BYOL (ResNet-50) Top 5 Accuracy 91.6% #85 of 144 Archive leaderboard report
Self-Supervised Person Re-Identification SYSU-30k BYOL Rank-1 12.7 #2 of 4 Archive leaderboard report
Semi-Supervised Image Classification ImageNet - 1% labeled data BYOL (ResNet-200 x2) Top 1 Accuracy 71.2% #18 of 65 Archive leaderboard report
Semi-Supervised Image Classification ImageNet - 1% labeled data BYOL (ResNet-200 x2) Top 5 Accuracy 89.5% #18 of 65 Archive leaderboard report
Semi-Supervised Image Classification ImageNet - 1% labeled data BYOL (ResNet-50 x4) Top 1 Accuracy 69.1% #23 of 65 Archive leaderboard report
Semi-Supervised Image Classification ImageNet - 1% labeled data BYOL (ResNet-50 x4) Top 5 Accuracy 87.9% #23 of 65 Archive leaderboard report
Semi-Supervised Image Classification ImageNet - 1% labeled data BYOL (ResNet-50 x2) Top 1 Accuracy 62.2% #36 of 65 Archive leaderboard report
Semi-Supervised Image Classification ImageNet - 1% labeled data BYOL (ResNet-50 x2) Top 5 Accuracy 84.1% #36 of 65 Archive leaderboard report
Semi-Supervised Image Classification ImageNet - 1% labeled data BYOL (ResNet-50) Top 1 Accuracy 53.2% #49 of 65 Archive leaderboard report
Semi-Supervised Image Classification ImageNet - 1% labeled data BYOL (ResNet-50) Top 5 Accuracy 78.4% #49 of 65 Archive leaderboard report

Ranks are positions in the archive's leaderboards as they stood at the 2025-07-28 snapshot. Results published since then are not among these rows, so a rank here is not a current standing.

Methods

1x1 ConvolutionAverage PoolingBatch NormalizationBottleneck Residual BlockConvolutionGlobal Average PoolingKaiming InitializationMax PoolingReLUResidual BlockResidual Connection

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections