{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/safegen-mitigating-unsafe-content-generation","title":"SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Models","arxiv_id":"2404.06666","date":"2024-04-10","proceeding":null,"authors":["Xinfeng Li","Yuchen Yang","Jiangyi Deng","Chen Yan","Yanjiao Chen","Xiaoyu Ji","Wenyuan Xu"],"abstract":"Text-to-image (T2I) models, such as Stable Diffusion, have exhibited remarkable performance in generating high-quality images from text descriptions in recent years. However, text-to-image models may be tricked into generating not-safe-for-work (NSFW) content, particularly in sexually explicit scenarios. Existing countermeasures mostly focus on filtering inappropriate inputs and outputs, or suppressing improper text embeddings, which can block sexually explicit content (e.g., naked) but may still be vulnerable to adversarial prompts -- inputs that appear innocent but are ill-intended. In this paper, we present SafeGen, a framework to mitigate sexual content generation by text-to-image models in a text-agnostic manner. The key idea is to eliminate explicit visual representations from the model regardless of the text input. In this way, the text-to-image model is resistant to adversarial prompts since such unsafe visual representations are obstructed from within. Extensive experiments conducted on four datasets and large-scale user studies demonstrate SafeGen's effectiveness in mitigating sexually explicit content generation while preserving the high-fidelity of benign images. SafeGen outperforms eight state-of-the-art baseline methods and achieves 99.4% sexual content removal performance. Furthermore, our constructed benchmark of adversarial prompts provides a basis for future development and evaluation of anti-NSFW-generation methods.","url_abs":"https://arxiv.org/abs/2404.06666v3","url_pdf":"https://arxiv.org/pdf/2404.06666v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"safegen-mitigating-unsafe-content-generation","repo_url":"https://github.com/letterligo/text-agnostic-governance","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"jax","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2404.06666","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2404.06666"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/letterligo/text-agnostic-governance","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran":4,"ran_fixture":1,"unverified":3},"by_repo_kind":{"official":{"samples":8,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"307ff0034e728f78","entry":"preprocess_video","repo":"letterligo/text-agnostic-governance","repo_kind":"official","path":"src/diffusers/pipelines/text_to_video_synthesis/pipeline_text_to_video_synth_img2img.py","file_url":"https://github.com/letterligo/text-agnostic-governance/blob/HEAD/src/diffusers/pipelines/text_to_video_synthesis/pipeline_text_to_video_synth_img2img.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"307ff0034e728f78"}},{"code_sha256_prefix":"c94fdebafdd5993d","entry":"rearrange_0","repo":"letterligo/text-agnostic-governance","repo_kind":"official","path":"src/diffusers/pipelines/text_to_video_synthesis/pipeline_text_to_video_zero.py","file_url":"https://github.com/letterligo/text-agnostic-governance/blob/HEAD/src/diffusers/pipelines/text_to_video_synthesis/pipeline_text_to_video_zero.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"c94fdebafdd5993d"}},{"code_sha256_prefix":"85ec51431a1b359b","entry":"rearrange_1","repo":"letterligo/text-agnostic-governance","repo_kind":"official","path":"src/diffusers/pipelines/text_to_video_synthesis/pipeline_text_to_video_zero.py","file_url":"https://github.com/letterligo/text-agnostic-governance/blob/HEAD/src/diffusers/pipelines/text_to_video_synthesis/pipeline_text_to_video_zero.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"85ec51431a1b359b"}},{"code_sha256_prefix":"55abb7c763ca4252","entry":"rearrange_3","repo":"letterligo/text-agnostic-governance","repo_kind":"official","path":"src/diffusers/pipelines/text_to_video_synthesis/pipeline_text_to_video_zero.py","file_url":"https://github.com/letterligo/text-agnostic-governance/blob/HEAD/src/diffusers/pipelines/text_to_video_synthesis/pipeline_text_to_video_zero.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"55abb7c763ca4252"}},{"code_sha256_prefix":"6abc8eb6537ace1e","entry":"tensor2vid","repo":"letterligo/text-agnostic-governance","repo_kind":"official","path":"src/diffusers/pipelines/text_to_video_synthesis/pipeline_text_to_video_synth.py","file_url":"https://github.com/letterligo/text-agnostic-governance/blob/HEAD/src/diffusers/pipelines/text_to_video_synthesis/pipeline_text_to_video_synth.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6abc8eb6537ace1e"}},{"code_sha256_prefix":"6cdb5efd1bc4e690","entry":"get_down_block","repo":"letterligo/text-agnostic-governance","repo_kind":"official","path":"src/diffusers/pipelines/versatile_diffusion/modeling_text_unet.py","file_url":"https://github.com/letterligo/text-agnostic-governance/blob/HEAD/src/diffusers/pipelines/versatile_diffusion/modeling_text_unet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"6cdb5efd1bc4e690"}},{"code_sha256_prefix":"052b20f9896c58b9","entry":"get_up_block","repo":"letterligo/text-agnostic-governance","repo_kind":"official","path":"src/diffusers/pipelines/versatile_diffusion/modeling_text_unet.py","file_url":"https://github.com/letterligo/text-agnostic-governance/blob/HEAD/src/diffusers/pipelines/versatile_diffusion/modeling_text_unet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"052b20f9896c58b9"}},{"code_sha256_prefix":"98bcd15cf273e3b4","entry":"retrieve_latents","repo":"letterligo/text-agnostic-governance","repo_kind":"official","path":"src/diffusers/pipelines/text_to_video_synthesis/pipeline_text_to_video_synth_img2img.py","file_url":"https://github.com/letterligo/text-agnostic-governance/blob/HEAD/src/diffusers/pipelines/text_to_video_synthesis/pipeline_text_to_video_synth_img2img.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"98bcd15cf273e3b4"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}