{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/towards-general-purpose-vision-systems","title":"Towards General Purpose Vision Systems","arxiv_id":"2104.00743","date":"2021-04-01","proceeding":null,"authors":["Tanmay Gupta","Amita Kamath","Aniruddha Kembhavi","Derek Hoiem"],"abstract":"Computer vision systems today are primarily N-purpose systems, designed and trained for a predefined set of tasks. Adapting such systems to new tasks is challenging and often requires non-trivial modifications to the network architecture (e.g. adding new output heads) or training process (e.g. adding new losses). To reduce the time and expertise required to develop new applications, we would like to create general purpose vision systems that can learn and perform a range of tasks without any modification to the architecture or learning process. In this paper, we propose GPV-1, a task-agnostic vision-language architecture that can learn and perform tasks that involve receiving an image and producing text and/or bounding boxes, including classification, localization, visual question answering, captioning, and more. We also propose evaluations of generality of architecture, skill-concept transfer, and learning efficiency that may inform future work on general purpose vision. Our experiments indicate GPV-1 is effective at multiple tasks, reuses some concept knowledge across tasks, can perform the Referring Expressions task zero-shot, and further improves upon the zero-shot performance using a few training samples.","url_abs":"https://arxiv.org/abs/2104.00743v2","url_pdf":"https://arxiv.org/pdf/2104.00743v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"towards-general-purpose-vision-systems","repo_url":"https://github.com/allenai/gpv-1","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}},{"paper_slug":"towards-general-purpose-vision-systems","repo_url":"https://github.com/mmaaz60/mvits_for_class_agnostic_od","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"visual-question-answering-1","task_name":"Visual Question Answering"},{"task_slug":"visual-question-answering","task_name":"Visual Question Answering (VQA)"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2104.00743","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2104.00743"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mmaaz60/mvits_for_class_agnostic_od","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/allenai/gpv-1","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"unverified":3},"by_repo_kind":{"official":{"samples":3,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"eaa5b6c9ca9995df","entry":"get_lrs","repo":"allenai/gpv-1","repo_kind":"official","path":"exp/gpv/finetune_distr.py","file_url":"https://github.com/allenai/gpv-1/blob/HEAD/exp/gpv/finetune_distr.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"eaa5b6c9ca9995df"}},{"code_sha256_prefix":"debfaa9338e2e751","entry":"grad_norm","repo":"allenai/gpv-1","repo_kind":"official","path":"exp/gpv/finetune_distr.py","file_url":"https://github.com/allenai/gpv-1/blob/HEAD/exp/gpv/finetune_distr.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"debfaa9338e2e751"}},{"code_sha256_prefix":"beead1a52916b495","entry":"vqa_accuracy","repo":"allenai/gpv-1","repo_kind":"official","path":"exp/gpv/metrics.py","file_url":"https://github.com/allenai/gpv-1/blob/HEAD/exp/gpv/metrics.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"beead1a52916b495"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}