{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2603-04803","title":"Guiding Diffusion-based Reconstruction with Contrastive Signals for Balanced Visual Representation","arxiv_id":"2603.04803","date":"2026-03-05","proceeding":null,"authors":["Boyu Han","Qianqian Xu","Shilong Bao","Zhiyong Yang","Ruochen Cui","Xilin Zhao","Qingming Huang"],"abstract":"The limited understanding capacity of the visual encoder in Contrastive Language-Image Pre-training (CLIP) has become a key bottleneck for downstream performance. This capacity includes both Discriminative Ability (D-Ability), which reflects class separability, and Detail Perceptual Ability (P-Ability), which focuses on fine-grained visual cues. Recent solutions use diffusion models to enhance representations by conditioning image reconstruction on CLIP visual tokens. We argue that such paradigms may compromise D-Ability and therefore fail to effectively address CLIP's representation limitations. To address this, we integrate contrastive signals into diffusion-based reconstruction to pursue more comprehensive visual representations. We begin with a straightforward design that augments the diffusion process with contrastive learning on input images. However, empirical results show that the naive combination suffers from gradient conflict and yields suboptimal performance. To balance the optimization, we introduce the Diffusion Contrastive Reconstruction (DCR), which unifies the learning objective. The key idea is to inject contrastive signals derived from each reconstructed image, rather than from the original input, into the diffusion process. Our theoretical analysis shows that the DCR loss can jointly optimize D-Ability and P-Ability. Extensive experiments across various benchmarks and multi-modal large language models validate the effectiveness of our method. The code is available at https://github.com/boyuh/DCR.","url_abs":"https://arxiv.org/abs/2603.04803","url_pdf":"https://arxiv.org/pdf/2603.04803","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2603.04803","atlas_url":"https://app.syntology.ai/?focus=2603.04803","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2603.04803"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/boyuh/DCR","reach":null}],"summary":{"unverified":7},"by_repo_kind":{"found_in_text":{"samples":7,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"5db80bffb9ac228f","entry":"benchmark_model","repo":"boyuh/DCR","repo_kind":"found_in_text","path":"evaluation/evaluate_mmvp_MetaCLIP_huge.py","file_url":"https://github.com/boyuh/DCR/blob/HEAD/evaluation/evaluate_mmvp_MetaCLIP_huge.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5db80bffb9ac228f"}},{"code_sha256_prefix":"7a8e61f1e0b30eda","entry":"benchmark_model","repo":"boyuh/DCR","repo_kind":"found_in_text","path":"evaluation/evaluate_mmvp_MetaCLIP_large.py","file_url":"https://github.com/boyuh/DCR/blob/HEAD/evaluation/evaluate_mmvp_MetaCLIP_large.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7a8e61f1e0b30eda"}},{"code_sha256_prefix":"e81743a997bc5d9a","entry":"load_clip_model_MetaCLIP","repo":"boyuh/DCR","repo_kind":"found_in_text","path":"clip_models/build_CLIP.py","file_url":"https://github.com/boyuh/DCR/blob/HEAD/clip_models/build_CLIP.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e81743a997bc5d9a"}},{"code_sha256_prefix":"5cb1fa8d536f97b8","entry":"load_clip_model_OpenAICLIP","repo":"boyuh/DCR","repo_kind":"found_in_text","path":"clip_models/build_CLIP.py","file_url":"https://github.com/boyuh/DCR/blob/HEAD/clip_models/build_CLIP.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"5cb1fa8d536f97b8"}},{"code_sha256_prefix":"ab135e852e338c55","entry":"load_clip_model_SigLIP","repo":"boyuh/DCR","repo_kind":"found_in_text","path":"clip_models/build_CLIP.py","file_url":"https://github.com/boyuh/DCR/blob/HEAD/clip_models/build_CLIP.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ab135e852e338c55"}},{"code_sha256_prefix":"d587d8888d060049","entry":"official_evaluation","repo":"boyuh/DCR","repo_kind":"found_in_text","path":"evaluation/evaluate_mmvp_MetaCLIP_huge.py","file_url":"https://github.com/boyuh/DCR/blob/HEAD/evaluation/evaluate_mmvp_MetaCLIP_huge.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d587d8888d060049"}},{"code_sha256_prefix":"4c9d81a8c8030f61","entry":"prepare_clip","repo":"boyuh/DCR","repo_kind":"found_in_text","path":"clip_models/sampling.py","file_url":"https://github.com/boyuh/DCR/blob/HEAD/clip_models/sampling.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4c9d81a8c8030f61"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.LG","source":"arxiv_2026.jsonl"},"syntology_extracted_results":{"kind":"leaderboard_placements","source":"Syntology's leaderboard-shaped extractor over the paper's own arXiv-HTML tables: a model pointed at a cell, the number was read from that cell and checked against the board's metric, dataset, split and scale, and an independent check accepted the entry; not reviewed by the paper's authors or the archive's editors","extractor_model":"global.anthropic.claude-sonnet-4-5-20250929-v1:0","verifier_model":"us.anthropic.claude-sonnet-4-5-20250929-v1:0","prompt_sha":"fa63d4bb9d755694","coverage":{"sentence":"Syntology has checked 6,885 of the 9,623 papers on this site that are newer than the archive; results from the others appear after they are checked.","papers_newer_than_archive":9623,"papers_checked":6885},"entries":[],"not_placed":{"boards":4,"rejected_by_independent_check":2,"refused_by_a_rule":2,"check_did_not_answer":0,"proposed_without_a_cell":0,"declined_by_site":0}}}