{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/online-bag-of-visual-words-generation-for","title":"OBoW: Online Bag-of-Visual-Words Generation for Self-Supervised Learning","arxiv_id":"2012.11552","date":"2020-12-21","proceeding":"CVPR 2021 1","authors":["Spyros Gidaris","Andrei Bursuc","Gilles Puy","Nikos Komodakis","Matthieu Cord","Patrick Pérez"],"abstract":"Learning image representations without human supervision is an important and active research field. Several recent approaches have successfully leveraged the idea of making such a representation invariant under different types of perturbations, especially via contrastive-based instance discrimination training. Although effective visual representations should indeed exhibit such invariances, there are other important characteristics, such as encoding contextual reasoning skills, for which alternative reconstruction-based approaches might be better suited. With this in mind, we propose a teacher-student scheme to learn representations by training a convolutional net to reconstruct a bag-of-visual-words (BoW) representation of an image, given as input a perturbed version of that same image. Our strategy performs an online training of both the teacher network (whose role is to generate the BoW targets) and the student network (whose role is to learn representations), along with an online update of the visual-words vocabulary (used for the BoW targets). This idea effectively enables fully online BoW-guided unsupervised learning. Extensive experiments demonstrate the interest of our BoW-based strategy which surpasses previous state-of-the-art methods (including contrastive-based ones) in several applications. For instance, in downstream tasks such Pascal object detection, Pascal classification and Places205 classification, our method improves over all prior unsupervised approaches, thus establishing new state-of-the-art results that are also significantly better even than those of supervised pre-training. We provide the implementation code at https://github.com/valeoai/obow.","url_abs":"https://arxiv.org/abs/2012.11552v2","url_pdf":"https://arxiv.org/pdf/2012.11552v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"online-bag-of-visual-words-generation-for","repo_url":"https://github.com/valeoai/obow","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"online-bag-of-visual-words-generation-for","repo_url":"https://github.com/TRAILab/ST-SLidR","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"online-bag-of-visual-words-generation-for","repo_url":"https://github.com/YufeiHU-fr/obow_ssl","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"unanswered"}}],"tasks":[{"task_slug":"object-detection","task_name":"Object Detection"},{"task_slug":"representation-learning","task_name":"Representation Learning"},{"task_slug":"self-supervised-image-classification","task_name":"Self-Supervised Image Classification"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"},{"task_slug":"semi-supervised-image-classification","task_name":"Semi-Supervised Image Classification"},{"task_slug":"unsupervised-pre-training","task_name":"Unsupervised Pre-training"},{"task_slug":"object-detection-1","task_name":"object-detection"}],"methods":[{"method_slug":"1x1-convolution","method_name":"1x1 Convolution"},{"method_slug":"average-pooling","method_name":"Average Pooling"},{"method_slug":"batch-normalization","method_name":"Batch Normalization"},{"method_slug":"bottleneck-residual-block","method_name":"Bottleneck Residual Block"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"global-average-pooling","method_name":"Global Average Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"residual-block","method_name":"Residual Block"},{"method_slug":"residual-connection","method_name":"Residual Connection"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/self-supervised-image-classification-on","task":"Self-Supervised Image Classification","dataset":"ImageNet","model":"OBoW (ResNet-50)","rank_in_archive_order":89,"of":144,"metrics":{"Number of Params":"24M","Top 1 Accuracy":"73.8%","Top 5 Accuracy":"92.2%"},"uses_additional_data":false},{"leaderboard":"/sota/semi-supervised-image-classification-on-1","task":"Semi-Supervised Image Classification","dataset":"ImageNet - 1% labeled data","model":"OBoW (ResNet-50)","rank_in_archive_order":54,"of":65,"metrics":{"Top 5 Accuracy":"82.9%"},"uses_additional_data":false},{"leaderboard":"/sota/semi-supervised-image-classification-on-2","task":"Semi-Supervised Image Classification","dataset":"ImageNet - 10% labeled data","model":"OBoW (ResNet-50)","rank_in_archive_order":52,"of":75,"metrics":{"Top 5 Accuracy":"90.7%"},"uses_additional_data":false}],"syntology":{"syntology_url":"https://syntology.ai/paper/2012.11552","atlas_url":"https://app.syntology.ai/?focus=2012.11552","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2012.11552"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/YufeiHU-fr/obow_ssl","reach":{"status":"unanswered"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/TRAILab/ST-SLidR","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/valeoai/obow","reach":null}],"summary":{"ran_draft_wrong":3},"by_repo_kind":{"official":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"fa31f8ff2b16e0a8","entry":"parse_model_opts","repo":"valeoai/obow","repo_kind":"official","path":"main_obow.py","file_url":"https://github.com/valeoai/obow/blob/HEAD/main_obow.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"fa31f8ff2b16e0a8"}},{"code_sha256_prefix":"522d73d740af361f","entry":"setup_model_distributed_data_parallel","repo":"valeoai/obow","repo_kind":"official","path":"main_obow.py","file_url":"https://github.com/valeoai/obow/blob/HEAD/main_obow.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"522d73d740af361f"}},{"code_sha256_prefix":"d10cef0e26695a9e","entry":"setup_model_for_distributed_training","repo":"valeoai/obow","repo_kind":"official","path":"main_linear_classification.py","file_url":"https://github.com/valeoai/obow/blob/HEAD/main_linear_classification.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NOASSERTION","inline_ok":false,"mcp_get_code":{"code_sha256":"d10cef0e26695a9e"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}