{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/learning-a-low-level-vision-generalist-via","title":"Learning A Low-Level Vision Generalist via Visual Task Prompt","arxiv_id":"2408.08601","date":"2024-08-16","proceeding":null,"authors":["Xiangyu Chen","Yihao Liu","Yuandong Pu","Wenlong Zhang","Jiantao Zhou","Yu Qiao","Chao Dong"],"abstract":"Building a unified model for general low-level vision tasks holds significant research and practical value. Current methods encounter several critical issues. Multi-task restoration approaches can address multiple degradation-to-clean restoration tasks, while their applicability to tasks with different target domains (e.g., image stylization) is limited. Methods like PromptGIP can handle multiple input-target domains but rely on the Masked Autoencoder (MAE) paradigm. Consequently, they are tied to the ViT architecture, resulting in suboptimal image reconstruction quality. In addition, these methods are sensitive to prompt image content and often struggle with low-frequency information processing. In this paper, we propose a Visual task Prompt-based Image Processing (VPIP) framework to overcome these challenges. VPIP employs visual task prompts to manage tasks with different input-target domains and allows flexible selection of backbone network suitable for general tasks. Besides, a new prompt cross-attention is introduced to facilitate interaction between the input and prompt information. Based on the VPIP framework, we train a low-level vision generalist model, namely GenLV, on 30 diverse tasks. Experimental results show that GenLV can successfully address a variety of low-level tasks, significantly outperforming existing methods both quantitatively and qualitatively. Codes are available at https://github.com/chxy95/GenLV.","url_abs":"https://arxiv.org/abs/2408.08601v1","url_pdf":"https://arxiv.org/pdf/2408.08601v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"learning-a-low-level-vision-generalist-via","repo_url":"https://github.com/chxy95/genlv","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"image-reconstruction","task_name":"Image Reconstruction"},{"task_slug":"image-stylization","task_name":"Image Stylization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2408.08601","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2408.08601"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/chxy95/genlv","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":1,"ran_draft_wrong":1,"ran_fixture":2,"unverified":6},"by_repo_kind":{"official":{"samples":10,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"d5ff9745fc569588","entry":"create_3_grid_from_np_images","repo":"chxy95/genlv","repo_kind":"official","path":"conference_version/evaluation_vit_prompt.py","file_url":"https://github.com/chxy95/genlv/blob/HEAD/conference_version/evaluation_vit_prompt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d5ff9745fc569588"}},{"code_sha256_prefix":"042da966a79899bf","entry":"expand_dim","repo":"chxy95/genlv","repo_kind":"official","path":"conference_version/models_xrestormer_prompt_crossattn_wores.py","file_url":"https://github.com/chxy95/genlv/blob/HEAD/conference_version/models_xrestormer_prompt_crossattn_wores.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"042da966a79899bf"}},{"code_sha256_prefix":"82a15cc1e46f7e4d","entry":"to_3d","repo":"chxy95/genlv","repo_kind":"official","path":"conference_version/models_xrestormer_prompt_crossattn_wores.py","file_url":"https://github.com/chxy95/genlv/blob/HEAD/conference_version/models_xrestormer_prompt_crossattn_wores.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"82a15cc1e46f7e4d"}},{"code_sha256_prefix":"b20f2a5df739a59e","entry":"to_4d","repo":"chxy95/genlv","repo_kind":"official","path":"conference_version/models_xrestormer_prompt_crossattn_wores.py","file_url":"https://github.com/chxy95/genlv/blob/HEAD/conference_version/models_xrestormer_prompt_crossattn_wores.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b20f2a5df739a59e"}},{"code_sha256_prefix":"98da48cfd7d11703","entry":"add_degradation_two_images","repo":"chxy95/genlv","repo_kind":"official","path":"conference_version/dataset/genlv_dataloader.py","file_url":"https://github.com/chxy95/genlv/blob/HEAD/conference_version/dataset/genlv_dataloader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"98da48cfd7d11703"}},{"code_sha256_prefix":"bfb65dbbd790d987","entry":"add_x_distortion_two_images","repo":"chxy95/genlv","repo_kind":"official","path":"conference_version/dataset/genlv_dataloader.py","file_url":"https://github.com/chxy95/genlv/blob/HEAD/conference_version/dataset/genlv_dataloader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bfb65dbbd790d987"}},{"code_sha256_prefix":"175aa529fb1ddac4","entry":"add_x_distortion_two_images","repo":"chxy95/genlv","repo_kind":"official","path":"extended_version/dataset/genlv_100_dataloader.py","file_url":"https://github.com/chxy95/genlv/blob/HEAD/extended_version/dataset/genlv_100_dataloader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"175aa529fb1ddac4"}},{"code_sha256_prefix":"ab5fc8a71e547c93","entry":"calculate_operators_two_images","repo":"chxy95/genlv","repo_kind":"official","path":"conference_version/dataset/genlv_dataloader.py","file_url":"https://github.com/chxy95/genlv/blob/HEAD/conference_version/dataset/genlv_dataloader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ab5fc8a71e547c93"}},{"code_sha256_prefix":"0330c0ccdad3ac5f","entry":"calculate_operators_two_images","repo":"chxy95/genlv","repo_kind":"official","path":"extended_version/dataset/genlv_100_dataloader.py","file_url":"https://github.com/chxy95/genlv/blob/HEAD/extended_version/dataset/genlv_100_dataloader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0330c0ccdad3ac5f"}},{"code_sha256_prefix":"fc065a68df58b44a","entry":"create_2_grid_from_np_images","repo":"chxy95/genlv","repo_kind":"official","path":"conference_version/evaluation_vit_prompt.py","file_url":"https://github.com/chxy95/genlv/blob/HEAD/conference_version/evaluation_vit_prompt.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fc065a68df58b44a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}