{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/arxiv-2605-08354","title":"Auto-Rubric as Reward: From Implicit Preferences to Explicit Multimodal Generative Criteria","arxiv_id":"2605.08354","date":"2026-05-08","proceeding":null,"authors":["Juanxi Tian","Fengyuan Liu","Jiaming Han","Yilei Jiang","Yongliang Wu","Yesheng Liu","Haodong Li","Furong Xu","Wanhua Li"],"abstract":"Aligning multimodal generative models with human preferences demands reward signals that respect the compositional, multi-dimensional structure of human judgment. Prevailing RLHF approaches reduce this structure to scalar or pairwise labels, collapsing nuanced preferences into opaque parametric proxies and exposing vulnerabilities to reward hacking. While recent Rubrics-as-Reward (RaR) methods attempt to recover this structure through explicit criteria, generating rubrics that are simultaneously reliable, scalable, and data-efficient remains an open problem. We introduce Auto-Rubric as Reward (ARR), a framework that reframes reward modeling from implicit weight optimization to explicit, criteria-based decomposition. Before any pairwise comparison, ARR externalizes a VLM's internalized preference knowledge as prompt-specific rubrics, translating holistic intent into independently verifiable quality dimensions. This conversion of implicit preference structure into inspectable, interpretable constraints substantially suppresses evaluation biases including positional bias, enabling both zero-shot deployment and few-shot conditioning on minimal supervision. To extend these gains into generative training, we propose Rubric Policy Optimization (RPO), which distills ARR's structured multi-dimensional evaluation into a robust binary reward, replacing opaque scalar regression with rubric-conditioned preference decisions that stabilize policy gradients. On text-to-image generation and image editing benchmarks, ARR-RPO outperforms pairwise reward models and VLM judges, demonstrating that explicitly externalizing implicit preference knowledge into structured rubrics achieves more reliable, data-efficient multimodal alignment, revealing that the bottleneck is the absence of a factorized interface, not a deficit of knowledge.","url_abs":"https://arxiv.org/abs/2605.08354","url_pdf":"https://arxiv.org/pdf/2605.08354","source":{"archive":null,"snapshot":"2025-07-28","note":"not in the Papers with Code archive (frozen at the snapshot)","row_kind":"graph","title_abstract_authors_date":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)"},"code_links":[],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2605.08354","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2605.08354"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"mentioned_in_github":null,"is_official":null,"provenance":"deterministic:regex_extraction","mentioned_in_paper":null,"url":"https://github.com/OpenEnvision/AutoRubric-as-Reward","reach":null}],"summary":{"ran_fixture":2,"ran_honours":3,"ran_draft_wrong":1,"ran_violates":1,"unverified":6},"by_repo_kind":{"found_in_text":{"samples":10,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":3,"samples":[{"code_sha256_prefix":"fe8a641669476f3e","entry":"flux_step","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"fe8a641669476f3e"}},{"code_sha256_prefix":"bfb49c26babf092b","entry":"gather_tensor","repo":"OpenEnvision/AutoRubric-as-Reward","repo_kind":"found_in_text","path":"fastvideo/train_rpo_flux.py","file_url":"https://github.com/OpenEnvision/AutoRubric-as-Reward/blob/HEAD/fastvideo/train_rpo_flux.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"bfb49c26babf092b"}},{"code_sha256_prefix":"cee2504594aabff2","entry":"get_arr_reward_sync","repo":"OpenEnvision/AutoRubric-as-Reward","repo_kind":"found_in_text","path":"fastvideo/train_rpo_flux.py","file_url":"https://github.com/OpenEnvision/AutoRubric-as-Reward/blob/HEAD/fastvideo/train_rpo_flux.py","link_basis":"first_harvest_node","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cee2504594aabff2"}},{"code_sha256_prefix":"c6e6dc7696a9dd53","entry":"pack_latents","repo":"OpenEnvision/AutoRubric-as-Reward","repo_kind":"found_in_text","path":"fastvideo/train_rpo_flux.py","file_url":"https://github.com/OpenEnvision/AutoRubric-as-Reward/blob/HEAD/fastvideo/train_rpo_flux.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c6e6dc7696a9dd53"}},{"code_sha256_prefix":"4080cb55d923dfba","entry":"prepare_latent_image_ids","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"4080cb55d923dfba"}},{"code_sha256_prefix":"a34c4529d41a7731","entry":"sd3_time_shift","repo":null,"repo_kind":null,"path":null,"file_url":null,"link_basis":"identical_code_first_harvested_elsewhere","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":null,"inline_ok":false,"mcp_get_code":{"code_sha256":"a34c4529d41a7731"}},{"code_sha256_prefix":"41cac86bce0769df","entry":"unpack_latents","repo":"OpenEnvision/AutoRubric-as-Reward","repo_kind":"found_in_text","path":"fastvideo/train_rpo_flux.py","file_url":"https://github.com/OpenEnvision/AutoRubric-as-Reward/blob/HEAD/fastvideo/train_rpo_flux.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"41cac86bce0769df"}},{"code_sha256_prefix":"a2781fb48dce4daa","entry":"assert_eq","repo":"OpenEnvision/AutoRubric-as-Reward","repo_kind":"found_in_text","path":"fastvideo/train_rpo_flux.py","file_url":"https://github.com/OpenEnvision/AutoRubric-as-Reward/blob/HEAD/fastvideo/train_rpo_flux.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a2781fb48dce4daa"}},{"code_sha256_prefix":"b019786f46692d16","entry":"disable_lora","repo":"OpenEnvision/AutoRubric-as-Reward","repo_kind":"found_in_text","path":"fastvideo/train_rpo_flux.py","file_url":"https://github.com/OpenEnvision/AutoRubric-as-Reward/blob/HEAD/fastvideo/train_rpo_flux.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"b019786f46692d16"}},{"code_sha256_prefix":"61c1d9feae808f1a","entry":"grpo_one_step","repo":"OpenEnvision/AutoRubric-as-Reward","repo_kind":"found_in_text","path":"fastvideo/train_rpo_flux.py","file_url":"https://github.com/OpenEnvision/AutoRubric-as-Reward/blob/HEAD/fastvideo/train_rpo_flux.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"61c1d9feae808f1a"}},{"code_sha256_prefix":"cbe8fb4b68e80866","entry":"run_sample_step","repo":"OpenEnvision/AutoRubric-as-Reward","repo_kind":"found_in_text","path":"fastvideo/train_rpo_flux.py","file_url":"https://github.com/OpenEnvision/AutoRubric-as-Reward/blob/HEAD/fastvideo/train_rpo_flux.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"cbe8fb4b68e80866"}},{"code_sha256_prefix":"5fd9b58e3fb41dc8","entry":"sample_reference_model","repo":"OpenEnvision/AutoRubric-as-Reward","repo_kind":"found_in_text","path":"fastvideo/train_rpo_flux.py","file_url":"https://github.com/OpenEnvision/AutoRubric-as-Reward/blob/HEAD/fastvideo/train_rpo_flux.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5fd9b58e3fb41dc8"}},{"code_sha256_prefix":"fa905e7d6e488d4f","entry":"train_one_step","repo":"OpenEnvision/AutoRubric-as-Reward","repo_kind":"found_in_text","path":"fastvideo/train_rpo_flux.py","file_url":"https://github.com/OpenEnvision/AutoRubric-as-Reward/blob/HEAD/fastvideo/train_rpo_flux.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fa905e7d6e488d4f"}}]},"arxiv_metadata":{"licence":"arXiv metadata, CC0 1.0 (https://info.arxiv.org/help/license)","fields":["title","abstract","authors","date"],"primary_category":"cs.AI","source":"arxiv_2026.jsonl"},"syntology_extracted_results":null}