{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/what-makes-good-examples-for-visual-in-1","title":"What Makes Good Examples for Visual In-Context Learning?","arxiv_id":"2301.13670","date":"2023-01-31","proceeding":"NeurIPS 2023 11","authors":["Yuanhan Zhang","Kaiyang Zhou","Ziwei Liu"],"abstract":"Large-scale models trained on broad data have recently become the mainstream architecture in computer vision due to their strong generalization performance. In this paper, the main focus is on an emergent ability in large vision models, known as in-context learning, which allows inference on unseen tasks by conditioning on in-context examples (a.k.a.~prompt) without updating the model parameters. This concept has been well-known in natural language processing but has only been studied very recently for large vision models. We for the first time provide a comprehensive investigation on the impact of in-context examples in computer vision, and find that the performance is highly sensitive to the choice of in-context examples. To overcome the problem, we propose a prompt retrieval framework to automate the selection of in-context examples. Specifically, we present (1) an unsupervised prompt retrieval method based on nearest example search using an off-the-shelf model, and (2) a supervised prompt retrieval method, which trains a neural network to choose examples that directly maximize in-context learning performance. The results demonstrate that our methods can bring non-trivial improvements to visual in-context learning in comparison to the commonly-used random selection.","url_abs":"https://arxiv.org/abs/2301.13670v2","url_pdf":"https://arxiv.org/pdf/2301.13670v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"what-makes-good-examples-for-visual-in-1","repo_url":"https://github.com/zhangyuanhan-ai/visual_prompt_retrieval","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"CC0-1.0"}}],"tasks":[{"task_slug":"in-context-learning","task_name":"In-Context Learning"},{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2301.13670","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2301.13670"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zhangyuanhan-ai/visual_prompt_retrieval","reach":{"status":"ok","spdx":"CC0-1.0"}}],"summary":{"ran_draft_wrong":1,"ran_fixture":1,"unverified":7},"by_repo_kind":{"official":{"samples":9,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"221b2d116fdf1032","entry":"get_obj_from_str","repo":"zhangyuanhan-ai/visual_prompt_retrieval","repo_kind":"official","path":"vqgan.py","file_url":"https://github.com/zhangyuanhan-ai/visual_prompt_retrieval/blob/HEAD/vqgan.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"CC0-1.0","inline_ok":true,"mcp_get_code":{"code_sha256":"221b2d116fdf1032"}},{"code_sha256_prefix":"cb49209c125de1b4","entry":"get_timestep_embedding","repo":"zhangyuanhan-ai/visual_prompt_retrieval","repo_kind":"official","path":"vqgan.py","file_url":"https://github.com/zhangyuanhan-ai/visual_prompt_retrieval/blob/HEAD/vqgan.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"CC0-1.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cb49209c125de1b4"}},{"code_sha256_prefix":"764136a5eff74434","entry":"calculate_metric","repo":"zhangyuanhan-ai/visual_prompt_retrieval","repo_kind":"official","path":"evaluate/evaluate_colorization.py","file_url":"https://github.com/zhangyuanhan-ai/visual_prompt_retrieval/blob/HEAD/evaluate/evaluate_colorization.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC0-1.0","inline_ok":true,"mcp_get_code":{"code_sha256":"764136a5eff74434"}},{"code_sha256_prefix":"69532fa60013e9f4","entry":"calculate_metric","repo":"zhangyuanhan-ai/visual_prompt_retrieval","repo_kind":"official","path":"evaluate/evaluate_reasoning.py","file_url":"https://github.com/zhangyuanhan-ai/visual_prompt_retrieval/blob/HEAD/evaluate/evaluate_reasoning.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC0-1.0","inline_ok":true,"mcp_get_code":{"code_sha256":"69532fa60013e9f4"}},{"code_sha256_prefix":"c68f71948cfa9d7c","entry":"get_demo_predictions","repo":"zhangyuanhan-ai/visual_prompt_retrieval","repo_kind":"official","path":"viz_utils.py","file_url":"https://github.com/zhangyuanhan-ai/visual_prompt_retrieval/blob/HEAD/viz_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC0-1.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c68f71948cfa9d7c"}},{"code_sha256_prefix":"8c8d20b40760059c","entry":"get_demo_predictions_with_mask","repo":"zhangyuanhan-ai/visual_prompt_retrieval","repo_kind":"official","path":"viz_utils.py","file_url":"https://github.com/zhangyuanhan-ai/visual_prompt_retrieval/blob/HEAD/viz_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC0-1.0","inline_ok":true,"mcp_get_code":{"code_sha256":"8c8d20b40760059c"}},{"code_sha256_prefix":"f0ac78b4e05be194","entry":"instantiate_from_config","repo":"zhangyuanhan-ai/visual_prompt_retrieval","repo_kind":"official","path":"vqgan.py","file_url":"https://github.com/zhangyuanhan-ai/visual_prompt_retrieval/blob/HEAD/vqgan.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC0-1.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f0ac78b4e05be194"}},{"code_sha256_prefix":"ce2513917f2cc4b5","entry":"is_square","repo":"zhangyuanhan-ai/visual_prompt_retrieval","repo_kind":"official","path":"evaluate/evaluate_reasoning.py","file_url":"https://github.com/zhangyuanhan-ai/visual_prompt_retrieval/blob/HEAD/evaluate/evaluate_reasoning.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC0-1.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ce2513917f2cc4b5"}},{"code_sha256_prefix":"ebd3af8bde9488f1","entry":"reverse_trans","repo":"zhangyuanhan-ai/visual_prompt_retrieval","repo_kind":"official","path":"tta.py","file_url":"https://github.com/zhangyuanhan-ai/visual_prompt_retrieval/blob/HEAD/tta.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"CC0-1.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ebd3af8bde9488f1"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}