{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/vision-superalignment-weak-to-strong","title":"Vision Superalignment: Weak-to-Strong Generalization for Vision Foundation Models","arxiv_id":"2402.03749","date":"2024-02-06","proceeding":null,"authors":["Jianyuan Guo","Hanting Chen","Chengcheng Wang","Kai Han","Chang Xu","Yunhe Wang"],"abstract":"Recent advancements in large language models have sparked interest in their extraordinary and near-superhuman capabilities, leading researchers to explore methods for evaluating and optimizing these abilities, which is called superalignment. In this context, our paper delves into the realm of vision foundation models, focusing on the concept of weak-to-strong generalization, which involves using a weaker model to supervise a stronger one, aiming to enhance the latter's capabilities beyond the former's limits. We introduce a novel and adaptively adjustable loss function for weak-to-strong supervision. Our comprehensive experiments span various scenarios, including few-shot learning, transfer learning, noisy label learning, and common knowledge distillation settings. The results are striking: our approach not only exceeds the performance benchmarks set by strong-to-strong generalization but also surpasses the outcomes of fine-tuning strong models with whole datasets. This compelling evidence underscores the significant potential of weak-to-strong generalization, showcasing its capability to substantially elevate the performance of vision foundation models. The code is available at https://github.com/ggjy/vision_weak_to_strong.","url_abs":"https://arxiv.org/abs/2402.03749v1","url_pdf":"https://arxiv.org/pdf/2402.03749v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"vision-superalignment-weak-to-strong","repo_url":"https://github.com/ggjy/vision_weak_to_strong","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"},{"task_slug":"knowledge-distillation","task_name":"Knowledge Distillation"},{"task_slug":"transfer-learning","task_name":"Transfer Learning"}],"methods":[{"method_slug":"knowledge-distillation","method_name":"Knowledge Distillation"},{"method_slug":"set","method_name":"SET"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2402.03749","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2402.03749"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ggjy/vision_weak_to_strong","reach":{"status":"ok"}}],"summary":{"ran_draft_wrong":2,"ran":2,"unverified":2},"by_repo_kind":{"official":{"samples":6,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":6,"samples":[{"code_sha256_prefix":"9d556d61294bb0a8","entry":"conv_block","repo":"ggjy/vision_weak_to_strong","repo_kind":"official","path":"few_shot_leaerning/models/convnet4.py","file_url":"https://github.com/ggjy/vision_weak_to_strong/blob/HEAD/few_shot_leaerning/models/convnet4.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9d556d61294bb0a8"}},{"code_sha256_prefix":"d8ee7a4b251eb3a4","entry":"get_layer_id_for_vit","repo":"ggjy/vision_weak_to_strong","repo_kind":"official","path":"classification/layer_decay.py","file_url":"https://github.com/ggjy/vision_weak_to_strong/blob/HEAD/classification/layer_decay.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d8ee7a4b251eb3a4"}},{"code_sha256_prefix":"591e8117f788fa7a","entry":"param_groups_lrd","repo":"ggjy/vision_weak_to_strong","repo_kind":"official","path":"classification/layer_decay.py","file_url":"https://github.com/ggjy/vision_weak_to_strong/blob/HEAD/classification/layer_decay.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"591e8117f788fa7a"}},{"code_sha256_prefix":"44c3aa4c610ea3f7","entry":"register","repo":"ggjy/vision_weak_to_strong","repo_kind":"official","path":"few_shot_leaerning/models/models.py","file_url":"https://github.com/ggjy/vision_weak_to_strong/blob/HEAD/few_shot_leaerning/models/models.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"44c3aa4c610ea3f7"}},{"code_sha256_prefix":"5bd23873a9c853d7","entry":"load","repo":"ggjy/vision_weak_to_strong","repo_kind":"official","path":"few_shot_leaerning/models/models.py","file_url":"https://github.com/ggjy/vision_weak_to_strong/blob/HEAD/few_shot_leaerning/models/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5bd23873a9c853d7"}},{"code_sha256_prefix":"232045c11773f4a3","entry":"make","repo":"ggjy/vision_weak_to_strong","repo_kind":"official","path":"few_shot_leaerning/models/models.py","file_url":"https://github.com/ggjy/vision_weak_to_strong/blob/HEAD/few_shot_leaerning/models/models.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"232045c11773f4a3"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}