{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/xpsr-cross-modal-priors-for-diffusion-based","title":"XPSR: Cross-modal Priors for Diffusion-based Image Super-Resolution","arxiv_id":"2403.05049","date":"2024-03-08","proceeding":null,"authors":["Yunpeng Qu","Kun Yuan","Kai Zhao","Qizhi Xie","Jinhua Hao","Ming Sun","Chao Zhou"],"abstract":"Diffusion-based methods, endowed with a formidable generative prior, have received increasing attention in Image Super-Resolution (ISR) recently. However, as low-resolution (LR) images often undergo severe degradation, it is challenging for ISR models to perceive the semantic and degradation information, resulting in restoration images with incorrect content or unrealistic artifacts. To address these issues, we propose a \\textit{Cross-modal Priors for Super-Resolution (XPSR)} framework. Within XPSR, to acquire precise and comprehensive semantic conditions for the diffusion model, cutting-edge Multimodal Large Language Models (MLLMs) are utilized. To facilitate better fusion of cross-modal priors, a \\textit{Semantic-Fusion Attention} is raised. To distill semantic-preserved information instead of undesired degradations, a \\textit{Degradation-Free Constraint} is attached between LR and its high-resolution (HR) counterpart. Quantitative and qualitative results show that XPSR is capable of generating high-fidelity and high-realism images across synthetic and real-world datasets. Codes are released at \\url{https://github.com/qyp2000/XPSR}.","url_abs":"https://arxiv.org/abs/2403.05049v2","url_pdf":"https://arxiv.org/pdf/2403.05049v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"xpsr-cross-modal-priors-for-diffusion-based","repo_url":"https://github.com/qyp2000/xpsr","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"image-super-resolution","task_name":"Image Super-Resolution"},{"task_slug":"super-resolution","task_name":"Super-Resolution"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2403.05049","atlas_url":"https://app.syntology.ai/?focus=2403.05049","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.05049"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/qyp2000/xpsr","reach":{"status":"ok"}},{"provenance":"deterministic:regex_extraction","url":"https://github.com/qyp2000/XPSR","reach":{"status":"ok"}}],"summary":{"ran":6,"ran_violates":1,"ran_honours":1,"ran_draft_wrong":1,"unverified":11},"by_repo_kind":{"official":{"samples":20,"ran":9,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":20,"samples":[{"code_sha256_prefix":"177c1cf8cd11be39","entry":"attn_forward_new_pt2_0","repo":"qyp2000/XPSR","repo_kind":"official","path":"myutils/vaehook.py","file_url":"https://github.com/qyp2000/XPSR/blob/HEAD/myutils/vaehook.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"177c1cf8cd11be39"}},{"code_sha256_prefix":"1b81186cfb3f8848","entry":"convert_image_to_fn","repo":"qyp2000/XPSR","repo_kind":"official","path":"myutils/img_util.py","file_url":"https://github.com/qyp2000/XPSR/blob/HEAD/myutils/img_util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"1b81186cfb3f8848"}},{"code_sha256_prefix":"608e364a9d2376a3","entry":"exists","repo":"qyp2000/XPSR","repo_kind":"official","path":"dataloader/localdataset.py","file_url":"https://github.com/qyp2000/XPSR/blob/HEAD/dataloader/localdataset.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"608e364a9d2376a3"}},{"code_sha256_prefix":"9910e2fc297f8665","entry":"identity","repo":"qyp2000/XPSR","repo_kind":"official","path":"myutils/misc.py","file_url":"https://github.com/qyp2000/XPSR/blob/HEAD/myutils/misc.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9910e2fc297f8665"}},{"code_sha256_prefix":"76a017e88938061d","entry":"inplace_nonlinearity","repo":"qyp2000/XPSR","repo_kind":"official","path":"myutils/vaehook.py","file_url":"https://github.com/qyp2000/XPSR/blob/HEAD/myutils/vaehook.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"76a017e88938061d"}},{"code_sha256_prefix":"bdd74d6bfd709ddf","entry":"rand_name","repo":"qyp2000/XPSR","repo_kind":"official","path":"myutils/misc.py","file_url":"https://github.com/qyp2000/XPSR/blob/HEAD/myutils/misc.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bdd74d6bfd709ddf"}},{"code_sha256_prefix":"73408b539d7e8eca","entry":"save_videos_grid","repo":"qyp2000/XPSR","repo_kind":"official","path":"myutils/img_util.py","file_url":"https://github.com/qyp2000/XPSR/blob/HEAD/myutils/img_util.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"73408b539d7e8eca"}},{"code_sha256_prefix":"bf260685de569a29","entry":"window_partition","repo":"qyp2000/XPSR","repo_kind":"official","path":"models/xpsr/unet_2d_blocks.py","file_url":"https://github.com/qyp2000/XPSR/blob/HEAD/models/xpsr/unet_2d_blocks.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"bf260685de569a29"}},{"code_sha256_prefix":"184db8216d8a8a05","entry":"window_reverse","repo":"qyp2000/XPSR","repo_kind":"official","path":"models/xpsr/unet_2d_blocks.py","file_url":"https://github.com/qyp2000/XPSR/blob/HEAD/models/xpsr/unet_2d_blocks.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"184db8216d8a8a05"}},{"code_sha256_prefix":"08822a10826682a0","entry":"attn_forward_new","repo":"qyp2000/XPSR","repo_kind":"official","path":"myutils/vaehook.py","file_url":"https://github.com/qyp2000/XPSR/blob/HEAD/myutils/vaehook.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"08822a10826682a0"}},{"code_sha256_prefix":"a1316b96fc0dc988","entry":"cond_cast_float","repo":"qyp2000/XPSR","repo_kind":"official","path":"myutils/devices.py","file_url":"https://github.com/qyp2000/XPSR/blob/HEAD/myutils/devices.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a1316b96fc0dc988"}},{"code_sha256_prefix":"6e1abd17fa1ce3b2","entry":"cond_cast_unet","repo":"qyp2000/XPSR","repo_kind":"official","path":"myutils/devices.py","file_url":"https://github.com/qyp2000/XPSR/blob/HEAD/myutils/devices.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"6e1abd17fa1ce3b2"}},{"code_sha256_prefix":"35fa1e7fbadd8f3d","entry":"convert_lora","repo":"qyp2000/XPSR","repo_kind":"official","path":"myutils/convert_lora_safetensor_to_diffusers.py","file_url":"https://github.com/qyp2000/XPSR/blob/HEAD/myutils/convert_lora_safetensor_to_diffusers.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"35fa1e7fbadd8f3d"}},{"code_sha256_prefix":"a861df384ac42eeb","entry":"get_device_for","repo":"qyp2000/XPSR","repo_kind":"official","path":"myutils/devices.py","file_url":"https://github.com/qyp2000/XPSR/blob/HEAD/myutils/devices.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a861df384ac42eeb"}},{"code_sha256_prefix":"794d4f7fc184e7e5","entry":"mesh_grid","repo":"qyp2000/XPSR","repo_kind":"official","path":"dataloader/degradations.py","file_url":"https://github.com/qyp2000/XPSR/blob/HEAD/dataloader/degradations.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"794d4f7fc184e7e5"}},{"code_sha256_prefix":"fce64ea4638101fc","entry":"opt_parse","repo":"qyp2000/XPSR","repo_kind":"official","path":"dataloader/realesrgan.py","file_url":"https://github.com/qyp2000/XPSR/blob/HEAD/dataloader/realesrgan.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fce64ea4638101fc"}},{"code_sha256_prefix":"321b9c73d589eb9e","entry":"pdf2","repo":"qyp2000/XPSR","repo_kind":"official","path":"dataloader/degradations.py","file_url":"https://github.com/qyp2000/XPSR/blob/HEAD/dataloader/degradations.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"321b9c73d589eb9e"}},{"code_sha256_prefix":"e3444ff2b9d24600","entry":"sigma_matrix2","repo":"qyp2000/XPSR","repo_kind":"official","path":"dataloader/degradations.py","file_url":"https://github.com/qyp2000/XPSR/blob/HEAD/dataloader/degradations.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e3444ff2b9d24600"}},{"code_sha256_prefix":"d40a55c91fa32c4f","entry":"window_process","repo":"qyp2000/XPSR","repo_kind":"official","path":"models/xpsr/unet_2d_blocks.py","file_url":"https://github.com/qyp2000/XPSR/blob/HEAD/models/xpsr/unet_2d_blocks.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d40a55c91fa32c4f"}},{"code_sha256_prefix":"da94debb8019ad46","entry":"zero_module","repo":"qyp2000/XPSR","repo_kind":"official","path":"models/xpsr/controlnet.py","file_url":"https://github.com/qyp2000/XPSR/blob/HEAD/models/xpsr/controlnet.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"da94debb8019ad46"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}