{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/rickrolling-the-artist-injecting-invisible","title":"Rickrolling the Artist: Injecting Backdoors into Text Encoders for Text-to-Image Synthesis","arxiv_id":"2211.02408","date":"2022-11-04","proceeding":"ICCV 2023 1","authors":["Lukas Struppek","Dominik Hintersdorf","Kristian Kersting"],"abstract":"While text-to-image synthesis currently enjoys great popularity among researchers and the general public, the security of these models has been neglected so far. Many text-guided image generation models rely on pre-trained text encoders from external sources, and their users trust that the retrieved models will behave as promised. Unfortunately, this might not be the case. We introduce backdoor attacks against text-guided generative models and demonstrate that their text encoders pose a major tampering risk. Our attacks only slightly alter an encoder so that no suspicious model behavior is apparent for image generations with clean prompts. By then inserting a single character trigger into the prompt, e.g., a non-Latin character or emoji, the adversary can trigger the model to either generate images with pre-defined attributes or images following a hidden, potentially malicious description. We empirically demonstrate the high effectiveness of our attacks on Stable Diffusion and highlight that the injection process of a single backdoor takes less than two minutes. Besides phrasing our approach solely as an attack, it can also force an encoder to forget phrases related to certain concepts, such as nudity or violence, and help to make image generation safer.","url_abs":"https://arxiv.org/abs/2211.02408v3","url_pdf":"https://arxiv.org/pdf/2211.02408v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"rickrolling-the-artist-injecting-invisible","repo_url":"https://github.com/lukasstruppek/rickrolling-the-artist","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"rickrolling-the-artist-injecting-invisible","repo_url":"https://github.com/lukasstruppek/exploiting-cultural-biases-via-homoglyphs","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"rickrolling-the-artist-injecting-invisible","repo_url":"https://github.com/lukasstruppek/the-biased-artist","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-generation","task_name":"Image Generation"}],"methods":[{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2211.02408","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2211.02408"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lukasstruppek/the-biased-artist","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lukasstruppek/rickrolling-the-artist","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/lukasstruppek/exploiting-cultural-biases-via-homoglyphs","reach":null}],"summary":{"ran_fixture":1,"ran_draft_wrong":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1},"listed":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"cff574411b733714","entry":"compute_text_embedding","repo":"lukasstruppek/the-biased-artist","repo_kind":"listed","path":"compute_weat.py","file_url":"https://github.com/lukasstruppek/the-biased-artist/blob/HEAD/compute_weat.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cff574411b733714"}},{"code_sha256_prefix":"7ca48a73ba523aa2","entry":"read_prompt_file","repo":"lukasstruppek/rickrolling-the-artist","repo_kind":"official","path":"generate_images.py","file_url":"https://github.com/lukasstruppek/rickrolling-the-artist/blob/HEAD/generate_images.py","link_basis":"first_harvest_node","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7ca48a73ba523aa2"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}