{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/diffusion-based-visual-anagram-as-multi-task","title":"Diffusion-based Visual Anagram as Multi-task Learning","arxiv_id":"2412.02693","date":"2024-12-03","proceeding":null,"authors":["Zhiyuan Xu","Yinhe Chen","Huan-ang Gao","Weiyan Zhao","Guiyu Zhang","Hao Zhao"],"abstract":"Visual anagrams are images that change appearance upon transformation, like flipping or rotation. With the advent of diffusion models, generating such optical illusions can be achieved by averaging noise across multiple views during the reverse denoising process. However, we observe two critical failure modes in this approach: (i) concept segregation, where concepts in different views are independently generated, which can not be considered a true anagram, and (ii) concept domination, where certain concepts overpower others. In this work, we cast the visual anagram generation problem in a multi-task learning setting, where different viewpoint prompts are analogous to different tasks,and derive denoising trajectories that align well across tasks simultaneously. At the core of our designed framework are two newly introduced techniques, where (i) an anti-segregation optimization strategy that promotes overlap in cross-attention maps between different concepts, and (ii) a noise vector balancing method that adaptively adjusts the influence of different tasks. Additionally, we observe that directly averaging noise predictions yields suboptimal performance because statistical properties may not be preserved, prompting us to derive a noise variance rectification method. Extensive qualitative and quantitative experiments demonstrate our method's superior ability to generate visual anagrams spanning diverse concepts.","url_abs":"https://arxiv.org/abs/2412.02693v1","url_pdf":"https://arxiv.org/pdf/2412.02693v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"diffusion-based-visual-anagram-as-multi-task","repo_url":"https://github.com/pixtella/anagram-mtl","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"Apache-2.0"}}],"tasks":[{"task_slug":"denoising","task_name":"Denoising"},{"task_slug":"multi-task-learning","task_name":"Multi-Task Learning"}],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"diffusion","method_name":"Diffusion"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2412.02693","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2412.02693"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/pixtella/anagram-mtl","reach":{"status":"ok","spdx":"Apache-2.0"}}],"summary":{"ran_honours":1,"ran_draft_wrong":1,"unverified":10},"by_repo_kind":{"official":{"samples":12,"ran":2,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"2ab2316ac6fdd869","entry":"betas_for_alpha_bar","repo":"pixtella/anagram-mtl","repo_kind":"official","path":"glide_text2im/gaussian_diffusion.py","file_url":"https://github.com/pixtella/anagram-mtl/blob/HEAD/glide_text2im/gaussian_diffusion.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"2ab2316ac6fdd869"}},{"code_sha256_prefix":"129b804760b3115f","entry":"zero_module","repo":"pixtella/anagram-mtl","repo_kind":"official","path":"glide_text2im/nn.py","file_url":"https://github.com/pixtella/anagram-mtl/blob/HEAD/glide_text2im/nn.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"129b804760b3115f"}},{"code_sha256_prefix":"ecd0fc28815b65ae","entry":"avg_pool_nd","repo":"pixtella/anagram-mtl","repo_kind":"official","path":"glide_text2im/nn.py","file_url":"https://github.com/pixtella/anagram-mtl/blob/HEAD/glide_text2im/nn.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"ecd0fc28815b65ae"}},{"code_sha256_prefix":"fe4eb545bbb728e0","entry":"conv_nd","repo":"pixtella/anagram-mtl","repo_kind":"official","path":"glide_text2im/nn.py","file_url":"https://github.com/pixtella/anagram-mtl/blob/HEAD/glide_text2im/nn.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"fe4eb545bbb728e0"}},{"code_sha256_prefix":"0ded220539a3a3ea","entry":"fetch_file_cached","repo":"pixtella/anagram-mtl","repo_kind":"official","path":"glide_text2im/download.py","file_url":"https://github.com/pixtella/anagram-mtl/blob/HEAD/glide_text2im/download.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0ded220539a3a3ea"}},{"code_sha256_prefix":"0996aa0f716afd2d","entry":"get_beta_schedule","repo":"pixtella/anagram-mtl","repo_kind":"official","path":"glide_text2im/gaussian_diffusion.py","file_url":"https://github.com/pixtella/anagram-mtl/blob/HEAD/glide_text2im/gaussian_diffusion.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0996aa0f716afd2d"}},{"code_sha256_prefix":"14e2c5e6c9d5ba6c","entry":"get_named_beta_schedule","repo":"pixtella/anagram-mtl","repo_kind":"official","path":"glide_text2im/gaussian_diffusion.py","file_url":"https://github.com/pixtella/anagram-mtl/blob/HEAD/glide_text2im/gaussian_diffusion.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"14e2c5e6c9d5ba6c"}},{"code_sha256_prefix":"0537243f2ce113e1","entry":"load_checkpoint","repo":"pixtella/anagram-mtl","repo_kind":"official","path":"glide_text2im/download.py","file_url":"https://github.com/pixtella/anagram-mtl/blob/HEAD/glide_text2im/download.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0537243f2ce113e1"}},{"code_sha256_prefix":"f4c92d5b3b58db03","entry":"noise_vector_rectification","repo":"pixtella/anagram-mtl","repo_kind":"official","path":"visual_anagrams/mtl.py","file_url":"https://github.com/pixtella/anagram-mtl/blob/HEAD/visual_anagrams/mtl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"f4c92d5b3b58db03"}},{"code_sha256_prefix":"0deefdc623c31f3d","entry":"register_attention_control","repo":"pixtella/anagram-mtl","repo_kind":"official","path":"attn_utils.py","file_url":"https://github.com/pixtella/anagram-mtl/blob/HEAD/attn_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"0deefdc623c31f3d"}},{"code_sha256_prefix":"9956e18fb3ff0e4a","entry":"sample_stage_1","repo":"pixtella/anagram-mtl","repo_kind":"official","path":"visual_anagrams/samplers_mtl.py","file_url":"https://github.com/pixtella/anagram-mtl/blob/HEAD/visual_anagrams/samplers_mtl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"9956e18fb3ff0e4a"}},{"code_sha256_prefix":"5d1a6c4919453f7c","entry":"sample_stage_2","repo":"pixtella/anagram-mtl","repo_kind":"official","path":"visual_anagrams/samplers_mtl.py","file_url":"https://github.com/pixtella/anagram-mtl/blob/HEAD/visual_anagrams/samplers_mtl.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"Apache-2.0","inline_ok":true,"mcp_get_code":{"code_sha256":"5d1a6c4919453f7c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}