{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/data-efficient-language-supervised-zero-shot-2","title":"Data Efficient Language-supervised Zero-shot Recognition with Optimal Transport Distillation","arxiv_id":"2112.09445","date":"2021-12-17","proceeding":"ICLR 2022 4","authors":["Bichen Wu","Ruizhe Cheng","Peizhao Zhang","Tianren Gao","Peter Vajda","Joseph E. Gonzalez"],"abstract":"Traditional computer vision models are trained to predict a fixed set of predefined categories. Recently, natural language has been shown to be a broader and richer source of supervision that provides finer descriptions to visual concepts than supervised \"gold\" labels. Previous works, such as CLIP, use InfoNCE loss to train a model to predict the pairing between images and text captions. CLIP, however, is data hungry and requires more than 400M image-text pairs for training. The inefficiency can be partially attributed to the fact that the image-text pairs are noisy. To address this, we propose OTTER (Optimal TransporT distillation for Efficient zero-shot Recognition), which uses online entropic optimal transport to find a soft image-text match as labels for contrastive learning. Based on pretrained image and text encoders, models trained with OTTER achieve strong performance with only 3M image text pairs. Compared with InfoNCE loss, label smoothing, and knowledge distillation, OTTER consistently outperforms these baselines in zero shot evaluation on Google Open Images (19,958 classes) and multi-labeled ImageNet 10K (10032 classes) from Tencent ML-Images. Over 42 evaluations on 7 different dataset/architecture settings x 6 metrics, OTTER outperforms (32) or ties (2) all baselines in 34 of them.","url_abs":"https://arxiv.org/abs/2112.09445v3","url_pdf":"https://arxiv.org/pdf/2112.09445v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"data-efficient-language-supervised-zero-shot-2","repo_url":"https://github.com/facebookresearch/otter","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"zero-shot-learning","task_name":"Zero-Shot Learning"}],"methods":[{"method_slug":"clip","method_name":"CLIP"},{"method_slug":"infonce","method_name":"InfoNCE"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2112.09445","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2112.09445"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/facebookresearch/otter","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":6},"by_repo_kind":{"official":{"samples":6,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"b40701674badc944","entry":"add_weight_decay","repo":"facebookresearch/otter","repo_kind":"official","path":"optim/optim_factory.py","file_url":"https://github.com/facebookresearch/otter/blob/HEAD/optim/optim_factory.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"b40701674badc944"}},{"code_sha256_prefix":"c1d6a1762ac426cb","entry":"build_optimizer","repo":"facebookresearch/otter","repo_kind":"official","path":"optim/optim_factory.py","file_url":"https://github.com/facebookresearch/otter/blob/HEAD/optim/optim_factory.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"c1d6a1762ac426cb"}},{"code_sha256_prefix":"9fbdbe606c3381f5","entry":"off_diag_avg","repo":"facebookresearch/otter","repo_kind":"official","path":"dataset_analysis.py","file_url":"https://github.com/facebookresearch/otter/blob/HEAD/dataset_analysis.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"9fbdbe606c3381f5"}},{"code_sha256_prefix":"d0761aacd8d0e3de","entry":"off_diag_max","repo":"facebookresearch/otter","repo_kind":"official","path":"dataset_analysis.py","file_url":"https://github.com/facebookresearch/otter/blob/HEAD/dataset_analysis.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"d0761aacd8d0e3de"}},{"code_sha256_prefix":"20ce03a3068a692a","entry":"on_diag_avg","repo":"facebookresearch/otter","repo_kind":"official","path":"dataset_analysis.py","file_url":"https://github.com/facebookresearch/otter/blob/HEAD/dataset_analysis.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"20ce03a3068a692a"}},{"code_sha256_prefix":"470b932b453c7bbe","entry":"sinkhorn","repo":"facebookresearch/otter","repo_kind":"official","path":"models/model_util.py","file_url":"https://github.com/facebookresearch/otter/blob/HEAD/models/model_util.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":false,"mcp_get_code":{"code_sha256":"470b932b453c7bbe"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}