{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2511-01678","title":"UniLumos: Fast and Unified Image and Video Relighting with Physics-Plausible Feedback","arxiv_id":"2511.01678","date":"2025-11-03","proceeding":"NeurIPS","authors":["Ropeway Liu","Hangjie Yuan","Bo Dong","Jiazheng Xing","Jinwang Wang","Rui Zhao","Yan Xing","Weihua Chen","Fan Wang"],"abstract":"Relighting is a crucial task with both practical demand and artistic value, and recent diffusion models have shown strong potential by enabling rich and controllable lighting effects. However, as they are typically optimized in semantic latent space, where proximity does not guarantee physical correctness in visual space, they often produce unrealistic results, such as overexposed highlights, misaligned shadows, and incorrect occlusions. We address this with UniLumos, a unified relighting framework for both images and videos that brings RGB-space geometry feedback into a flow matching backbone. By supervising the model with depth and normal maps extracted from its outputs, we explicitly align lighting effects with the scene structure, enhancing physical plausibility. Nevertheless, this feedback requires high-quality outputs for supervision in visual space, making standard multi-step denoising computationally expensive. To mitigate this, we employ path consistency learning, allowing supervision to remain effective even under few-step training regimes. To enable fine-grained relighting control and supervision, we design a structured six-dimensional annotation protocol capturing core illumination attributes. Building upon this, we propose LumosBench, a disentangled attribute-level benchmark that evaluates lighting controllability via large vision-language models, enabling automatic and interpretable assessment of relighting precision across individual dimensions. Extensive experiments demonstrate that UniLumos achieves state-of-the-art relighting quality with significantly improved physical consistency, while delivering a 20x speedup for both image and video relighting. Code is available at https://github.com/alibaba-damo-academy/Lumos-Custom.","url_abs":"https://arxiv.org/abs/2511.01678","url_pdf":"https://arxiv.org/pdf/2511.01678","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2511.01678","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2511.01678"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/alibaba-damo-academy/Lumos-Custom","reach":null}],"summary":{"ran_fixture":3},"by_repo_kind":{"found_in_text":{"samples":3,"ran":3,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"f051ed32519849b2","entry":"mae_loss","repo":"alibaba-damo-academy/Lumos-Custom","repo_kind":"found_in_text","path":"UniLumos/UniLumos/src/schedulers/rectified_flow.py","file_url":"https://github.com/alibaba-damo-academy/Lumos-Custom/blob/HEAD/UniLumos/UniLumos/src/schedulers/rectified_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"f051ed32519849b2"}},{"code_sha256_prefix":"18f75e876d5831d4","entry":"patchify","repo":"alibaba-damo-academy/Lumos-Custom","repo_kind":"found_in_text","path":"UniLumos/UniLumos/src/schedulers/rectified_flow.py","file_url":"https://github.com/alibaba-damo-academy/Lumos-Custom/blob/HEAD/UniLumos/UniLumos/src/schedulers/rectified_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"18f75e876d5831d4"}},{"code_sha256_prefix":"476e9e8e2e851afa","entry":"xpadding","repo":"alibaba-damo-academy/Lumos-Custom","repo_kind":"found_in_text","path":"UniLumos/UniLumos/src/schedulers/rectified_flow.py","file_url":"https://github.com/alibaba-damo-academy/Lumos-Custom/blob/HEAD/UniLumos/UniLumos/src/schedulers/rectified_flow.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"476e9e8e2e851afa"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.CV","source":"arxiv_api"},"syntology_extracted_results":null}