{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/boxdiff-text-to-image-synthesis-with-training","title":"BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusion","arxiv_id":"2307.10816","date":"2023-07-20","proceeding":"ICCV 2023 1","authors":["Jinheng Xie","Yuexiang Li","Yawen Huang","Haozhe Liu","Wentian Zhang","Yefeng Zheng","Mike Zheng Shou"],"abstract":"Recent text-to-image diffusion models have demonstrated an astonishing capacity to generate high-quality images. However, researchers mainly studied the way of synthesizing images with only text prompts. While some works have explored using other modalities as conditions, considerable paired data, e.g., box/mask-image pairs, and fine-tuning time are required for nurturing models. As such paired data is time-consuming and labor-intensive to acquire and restricted to a closed set, this potentially becomes the bottleneck for applications in an open world. This paper focuses on the simplest form of user-provided conditions, e.g., box or scribble. To mitigate the aforementioned problem, we propose a training-free method to control objects and contexts in the synthesized images adhering to the given spatial conditions. Specifically, three spatial constraints, i.e., Inner-Box, Outer-Box, and Corner Constraints, are designed and seamlessly integrated into the denoising step of diffusion models, requiring no additional training and massive annotated layout data. Extensive experimental results demonstrate that the proposed constraints can control what and where to present in the images while retaining the ability of Diffusion models to synthesize with high fidelity and diverse concept coverage. The code is publicly available at https://github.com/showlab/BoxDiff.","url_abs":"https://arxiv.org/abs/2307.10816v4","url_pdf":"https://arxiv.org/pdf/2307.10816v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"boxdiff-text-to-image-synthesis-with-training","repo_url":"https://github.com/showlab/boxdiff","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}},{"paper_slug":"boxdiff-text-to-image-synthesis-with-training","repo_url":"https://github.com/sierkinhane/boxdiff","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok"}}],"tasks":[{"task_slug":"conditional-text-to-image-synthesis","task_name":"Conditional Text-to-Image Synthesis"},{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"image-generation","task_name":"Image Generation"},{"task_slug":"text-to-image-generation","task_name":"Text-to-Image Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[{"leaderboard":"/sota/conditional-text-to-image-synthesis-on-coco-1","task":"Conditional Text-to-Image Synthesis","dataset":"COCO-MIG","model":"Box-Diffusion (zero-shot)","rank_in_archive_order":5,"of":5,"metrics":{"instance success rate":"0.16","mIoU":"0.26"},"uses_additional_data":false}],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2307.10816","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.10816"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/sierkinhane/boxdiff","reach":{"status":"ok"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/showlab/boxdiff","reach":{"status":"ok"}}],"summary":{"ran":3,"ran_honours":1,"unverified":1},"by_repo_kind":{"official":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"3cfdfcac97b762af","entry":"aggregate_attention","repo":"showlab/boxdiff","repo_kind":"official","path":"utils/ptp_utils.py","file_url":"https://github.com/showlab/boxdiff/blob/HEAD/utils/ptp_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3cfdfcac97b762af"}},{"code_sha256_prefix":"0078d117f74b7c20","entry":"get_image_grid","repo":"showlab/boxdiff","repo_kind":"official","path":"utils/vis_utils.py","file_url":"https://github.com/showlab/boxdiff/blob/HEAD/utils/vis_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"0078d117f74b7c20"}},{"code_sha256_prefix":"e9f575264129ec46","entry":"get_indices_to_alter","repo":"showlab/boxdiff","repo_kind":"official","path":"run_gligen_boxdiff.py","file_url":"https://github.com/showlab/boxdiff/blob/HEAD/run_gligen_boxdiff.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"e9f575264129ec46"}},{"code_sha256_prefix":"3c75935b4fdb2c79","entry":"show_image_relevance","repo":"showlab/boxdiff","repo_kind":"official","path":"utils/vis_utils.py","file_url":"https://github.com/showlab/boxdiff/blob/HEAD/utils/vis_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"3c75935b4fdb2c79"}},{"code_sha256_prefix":"42f35256bd885673","entry":"text_under_image","repo":"showlab/boxdiff","repo_kind":"official","path":"utils/ptp_utils.py","file_url":"https://github.com/showlab/boxdiff/blob/HEAD/utils/ptp_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"42f35256bd885673"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}