{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/unirl-self-improving-unified-multimodal","title":"UniRL: Self-Improving Unified Multimodal Models via Supervised and Reinforcement Learning","arxiv_id":"2505.23380","date":"2025-05-29","proceeding":null,"authors":["Weijia Mao","Zhenheng Yang","Mike Zheng Shou"],"abstract":"Unified multimodal large language models such as Show-o and Janus have achieved strong performance across both generation and understanding tasks. However, these models typically rely on large-scale datasets and require substantial computation during the pretraining stage. In addition, several post-training methods have been proposed, but they often depend on external data or are limited to task-specific customization. In this work, we introduce UniRL, a self-improving post-training approach. Our approach enables the model to generate images from prompts and use them as training data in each iteration, without relying on any external image data. Moreover, it enables the two tasks to enhance each other: the generated images are used for understanding, and the understanding results are used to supervise generation. We explore supervised fine-tuning (SFT) and Group Relative Policy Optimization (GRPO) to optimize the models. UniRL offers three key advantages: (1) it requires no external image data, as all training samples are generated by the model itself during training; (2) it not only improves individual task performance, but also reduces the imbalance between generation and understanding; and (3) it requires only several additional training steps during the post-training stage. We evaluate UniRL on top of Show-o and Janus, achieving a GenEval score of 0.77 for Show-o and 0.65 for Janus. Code and models will be released in https://github.com/showlab/UniRL.","url_abs":"https://arxiv.org/abs/2505.23380v1","url_pdf":"https://arxiv.org/pdf/2505.23380v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"unirl-self-improving-unified-multimodal","repo_url":"https://github.com/showlab/unirl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"jax","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2505.23380","atlas_url":"https://app.syntology.ai/?focus=2505.23380","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.23380"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/showlab/unirl","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_draft_wrong":4,"ran":6,"unverified":2},"by_repo_kind":{"official":{"samples":12,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c3a6b977022957cb","entry":"Normalize","repo":"showlab/unirl","repo_kind":"official","path":"models/common_modules.py","file_url":"https://github.com/showlab/unirl/blob/HEAD/models/common_modules.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c3a6b977022957cb"}},{"code_sha256_prefix":"2f96ce289cb03a29","entry":"compute_iou","repo":"showlab/unirl","repo_kind":"official","path":"evaluate_imgs.py","file_url":"https://github.com/showlab/unirl/blob/HEAD/evaluate_imgs.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2f96ce289cb03a29"}},{"code_sha256_prefix":"d15d620fa9ed83c7","entry":"flatten_omega_conf","repo":"showlab/unirl","repo_kind":"official","path":"inference.py","file_url":"https://github.com/showlab/unirl/blob/HEAD/inference.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d15d620fa9ed83c7"}},{"code_sha256_prefix":"a8366f238618edad","entry":"get_constant_schedule","repo":"showlab/unirl","repo_kind":"official","path":"models/lr_schedulers.py","file_url":"https://github.com/showlab/unirl/blob/HEAD/models/lr_schedulers.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"a8366f238618edad"}},{"code_sha256_prefix":"11f07821d1b2286b","entry":"get_constant_schedule_with_warmup","repo":"showlab/unirl","repo_kind":"official","path":"models/lr_schedulers.py","file_url":"https://github.com/showlab/unirl/blob/HEAD/models/lr_schedulers.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"11f07821d1b2286b"}},{"code_sha256_prefix":"1975e0c51e6f5753","entry":"get_linear_schedule_with_warmup","repo":"showlab/unirl","repo_kind":"official","path":"models/lr_schedulers.py","file_url":"https://github.com/showlab/unirl/blob/HEAD/models/lr_schedulers.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"1975e0c51e6f5753"}},{"code_sha256_prefix":"3137073275f8c21a","entry":"nonlinearity","repo":"showlab/unirl","repo_kind":"official","path":"models/common_modules.py","file_url":"https://github.com/showlab/unirl/blob/HEAD/models/common_modules.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"3137073275f8c21a"}},{"code_sha256_prefix":"d8437c41cc2a860f","entry":"soft_target_cross_entropy","repo":"showlab/unirl","repo_kind":"official","path":"inference.py","file_url":"https://github.com/showlab/unirl/blob/HEAD/inference.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"d8437c41cc2a860f"}},{"code_sha256_prefix":"844a4aecea80d106","entry":"timed","repo":"showlab/unirl","repo_kind":"official","path":"evaluate_imgs.py","file_url":"https://github.com/showlab/unirl/blob/HEAD/evaluate_imgs.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"844a4aecea80d106"}},{"code_sha256_prefix":"2d82abbef2c4addb","entry":"unpack_time","repo":"showlab/unirl","repo_kind":"official","path":"models/common_modules.py","file_url":"https://github.com/showlab/unirl/blob/HEAD/models/common_modules.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2d82abbef2c4addb"}},{"code_sha256_prefix":"376bf22d5737fc2a","entry":"get_logger","repo":"showlab/unirl","repo_kind":"official","path":"models/logging.py","file_url":"https://github.com/showlab/unirl/blob/HEAD/models/logging.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"376bf22d5737fc2a"}},{"code_sha256_prefix":"509a71c867af544b","entry":"get_vq_model_class","repo":"showlab/unirl","repo_kind":"official","path":"inference.py","file_url":"https://github.com/showlab/unirl/blob/HEAD/inference.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"509a71c867af544b"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}