{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/mj-bench-is-your-multimodal-reward-model","title":"MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?","arxiv_id":"2407.04842","date":"2024-07-05","proceeding":null,"authors":["Zhaorun Chen","Yichao Du","Zichen Wen","Yiyang Zhou","Chenhang Cui","Zhenzhen Weng","Haoqin Tu","Chaoqi Wang","Zhengwei Tong","Qinglan Huang","Canyu Chen","Qinghao Ye","Zhihong Zhu","Yuqing Zhang","Jiawei Zhou","Zhuokai Zhao","Rafael Rafailov","Chelsea Finn","Huaxiu Yao"],"abstract":"While text-to-image models like DALLE-3 and Stable Diffusion are rapidly proliferating, they often encounter challenges such as hallucination, bias, and the production of unsafe, low-quality output. To effectively address these issues, it is crucial to align these models with desired behaviors based on feedback from a multimodal judge. Despite their significance, current multimodal judges frequently undergo inadequate evaluation of their capabilities and limitations, potentially leading to misalignment and unsafe fine-tuning outcomes. To address this issue, we introduce MJ-Bench, a novel benchmark which incorporates a comprehensive preference dataset to evaluate multimodal judges in providing feedback for image generation models across four key perspectives: alignment, safety, image quality, and bias. Specifically, we evaluate a large variety of multimodal judges including smaller-sized CLIP-based scoring models, open-source VLMs (e.g. LLaVA family), and close-source VLMs (e.g. GPT-4o, Claude 3) on each decomposed subcategory of our preference dataset. Experiments reveal that close-source VLMs generally provide better feedback, with GPT-4o outperforming other judges in average. Compared with open-source VLMs, smaller-sized scoring models can provide better feedback regarding text-image alignment and image quality, while VLMs provide more accurate feedback regarding safety and generation bias due to their stronger reasoning capabilities. Further studies in feedback scale reveal that VLM judges can generally provide more accurate and stable feedback in natural language (Likert-scale) than numerical scales. Notably, human evaluations on end-to-end fine-tuned models using separate feedback from these multimodal judges provide similar conclusions, further confirming the effectiveness of MJ-Bench. All data, code, models are available at https://huggingface.co/MJ-Bench.","url_abs":"https://arxiv.org/abs/2407.04842v1","url_pdf":"https://arxiv.org/pdf/2407.04842v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"mj-bench-is-your-multimodal-reward-model","repo_url":"https://github.com/MJ-Bench/MJ-Bench","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"hallucination","task_name":"Hallucination"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"text-to-image-generation-1","task_name":"Text to Image Generation"},{"task_slug":"text-to-image-generation","task_name":"Text-to-Image Generation"}],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2407.04842","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2407.04842"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/MJ-Bench/MJ-Bench","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":11,"unverified":3},"by_repo_kind":{"official":{"samples":14,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"ec77e2763d61573c","entry":"check_demographic_group","repo":"MJ-Bench/MJ-Bench","repo_kind":"official","path":"finetune_datasets/bias_dataset_select.py","file_url":"https://github.com/MJ-Bench/MJ-Bench/blob/HEAD/finetune_datasets/bias_dataset_select.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ec77e2763d61573c"}},{"code_sha256_prefix":"52cb1c4da16603c7","entry":"extract_first_number","repo":"MJ-Bench/MJ-Bench","repo_kind":"official","path":"evaluate_bias_avg.py","file_url":"https://github.com/MJ-Bench/MJ-Bench/blob/HEAD/evaluate_bias_avg.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"52cb1c4da16603c7"}},{"code_sha256_prefix":"bf0c454d7dfe9150","entry":"extract_narrative_score","repo":"MJ-Bench/MJ-Bench","repo_kind":"official","path":"evaluate_bias_avg.py","file_url":"https://github.com/MJ-Bench/MJ-Bench/blob/HEAD/evaluate_bias_avg.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bf0c454d7dfe9150"}},{"code_sha256_prefix":"8957ed6aa5f44109","entry":"get_dirs","repo":"MJ-Bench/MJ-Bench","repo_kind":"official","path":"bias/get_finetune_prompt.py","file_url":"https://github.com/MJ-Bench/MJ-Bench/blob/HEAD/bias/get_finetune_prompt.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8957ed6aa5f44109"}},{"code_sha256_prefix":"113f20f07f1ceea2","entry":"get_label","repo":"MJ-Bench/MJ-Bench","repo_kind":"official","path":"experimental/ddpo_finetune_with_vlm.py","file_url":"https://github.com/MJ-Bench/MJ-Bench/blob/HEAD/experimental/ddpo_finetune_with_vlm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"113f20f07f1ceea2"}},{"code_sha256_prefix":"17ad7536db5b4682","entry":"get_pred","repo":"MJ-Bench/MJ-Bench","repo_kind":"official","path":"experimental/ddpo_finetune_with_vlm.py","file_url":"https://github.com/MJ-Bench/MJ-Bench/blob/HEAD/experimental/ddpo_finetune_with_vlm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"17ad7536db5b4682"}},{"code_sha256_prefix":"cc893884872e9eda","entry":"load_json_dataset","repo":"MJ-Bench/MJ-Bench","repo_kind":"official","path":"inference.py","file_url":"https://github.com/MJ-Bench/MJ-Bench/blob/HEAD/inference.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cc893884872e9eda"}},{"code_sha256_prefix":"4fb4a653fefe74ee","entry":"load_prompt_template","repo":"MJ-Bench/MJ-Bench","repo_kind":"official","path":"inference.py","file_url":"https://github.com/MJ-Bench/MJ-Bench/blob/HEAD/inference.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4fb4a653fefe74ee"}},{"code_sha256_prefix":"cb76eb23e440636a","entry":"open_image","repo":"MJ-Bench/MJ-Bench","repo_kind":"official","path":"experimental/ddpo_finetune_with_vlm.py","file_url":"https://github.com/MJ-Bench/MJ-Bench/blob/HEAD/experimental/ddpo_finetune_with_vlm.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cb76eb23e440636a"}},{"code_sha256_prefix":"54ca6c2611c3c399","entry":"prompt_fn","repo":"MJ-Bench/MJ-Bench","repo_kind":"official","path":"experimental/ddpo_finetune.py","file_url":"https://github.com/MJ-Bench/MJ-Bench/blob/HEAD/experimental/ddpo_finetune.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"54ca6c2611c3c399"}},{"code_sha256_prefix":"eb28e9ec2502d447","entry":"tokenize_captions","repo":"MJ-Bench/MJ-Bench","repo_kind":"official","path":"experimental/dpo_finetune.py","file_url":"https://github.com/MJ-Bench/MJ-Bench/blob/HEAD/experimental/dpo_finetune.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eb28e9ec2502d447"}},{"code_sha256_prefix":"1b1556b0ebaba73c","entry":"fit_gaussian_and_report_properties","repo":"MJ-Bench/MJ-Bench","repo_kind":"official","path":"evaluate_bias_type.py","file_url":"https://github.com/MJ-Bench/MJ-Bench/blob/HEAD/evaluate_bias_type.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1b1556b0ebaba73c"}},{"code_sha256_prefix":"1f631e56c6e042ac","entry":"import_model_class_from_model_name_or_path","repo":"MJ-Bench/MJ-Bench","repo_kind":"official","path":"experimental/dpo_finetune.py","file_url":"https://github.com/MJ-Bench/MJ-Bench/blob/HEAD/experimental/dpo_finetune.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1f631e56c6e042ac"}},{"code_sha256_prefix":"38e20568bab6256d","entry":"parse_args","repo":"MJ-Bench/MJ-Bench","repo_kind":"official","path":"experimental/dpo_finetune.py","file_url":"https://github.com/MJ-Bench/MJ-Bench/blob/HEAD/experimental/dpo_finetune.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"38e20568bab6256d"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}