{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/synthesizing-artistic-cinemagraphs-from-text","title":"Text-Guided Synthesis of Eulerian Cinemagraphs","arxiv_id":"2307.03190","date":"2023-07-06","proceeding":null,"authors":["Aniruddha Mahapatra","Aliaksandr Siarohin","Hsin-Ying Lee","Sergey Tulyakov","Jun-Yan Zhu"],"abstract":"We introduce Text2Cinemagraph, a fully automated method for creating cinemagraphs from text descriptions - an especially challenging task when prompts feature imaginary elements and artistic styles, given the complexity of interpreting the semantics and motions of these images. We focus on cinemagraphs of fluid elements, such as flowing rivers, and drifting clouds, which exhibit continuous motion and repetitive textures. Existing single-image animation methods fall short on artistic inputs, and recent text-based video methods frequently introduce temporal inconsistencies, struggling to keep certain regions static. To address these challenges, we propose an idea of synthesizing image twins from a single text prompt - a pair of an artistic image and its pixel-aligned corresponding natural-looking twin. While the artistic image depicts the style and appearance detailed in our text prompt, the realistic counterpart greatly simplifies layout and motion analysis. Leveraging existing natural image and video datasets, we can accurately segment the realistic image and predict plausible motion given the semantic information. The predicted motion can then be transferred to the artistic image to create the final cinemagraph. Our method outperforms existing approaches in creating cinemagraphs for natural landscapes as well as artistic and other-worldly scenes, as validated by automated metrics and user studies. Finally, we demonstrate two extensions: animating existing paintings and controlling motion directions using text.","url_abs":"https://arxiv.org/abs/2307.03190v3","url_pdf":"https://arxiv.org/pdf/2307.03190v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"synthesizing-artistic-cinemagraphs-from-text","repo_url":"https://github.com/text2cinemagraph/text2cinemagraph","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"image-animation","task_name":"Image Animation"}],"methods":[{"method_slug":"focus","method_name":"Focus"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2307.03190","atlas_url":"https://app.syntology.ai/?focus=2307.03190","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2307.03190"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/text2cinemagraph/text2cinemagraph","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_violates":2,"ran":5,"ran_honours":1,"ran_draft_wrong":3,"unverified":2},"by_repo_kind":{"official":{"samples":13,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"424012cb37b31172","entry":"default","repo":"text2cinemagraph/text2cinemagraph","repo_kind":"official","path":"models/attention.py","file_url":"https://github.com/text2cinemagraph/text2cinemagraph/blob/HEAD/models/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"well_formed","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"424012cb37b31172"}},{"code_sha256_prefix":"aa5486a3650902d8","entry":"exists","repo":"text2cinemagraph/text2cinemagraph","repo_kind":"official","path":"models/attention.py","file_url":"https://github.com/text2cinemagraph/text2cinemagraph/blob/HEAD/models/attention.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"aa5486a3650902d8"}},{"code_sha256_prefix":"fca998c0c502f6dc","entry":"find_dir_from_caption","repo":"text2cinemagraph/text2cinemagraph","repo_kind":"official","path":"inference_t2c.py","file_url":"https://github.com/text2cinemagraph/text2cinemagraph/blob/HEAD/inference_t2c.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fca998c0c502f6dc"}},{"code_sha256_prefix":"d7c46b2392abf5fd","entry":"fused_bn","repo":"text2cinemagraph/text2cinemagraph","repo_kind":"official","path":"models/normalization.py","file_url":"https://github.com/text2cinemagraph/text2cinemagraph/blob/HEAD/models/normalization.py","link_basis":"plan_row","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d7c46b2392abf5fd"}},{"code_sha256_prefix":"ef64c9bacc1b00b0","entry":"get_D_norm_layer","repo":"text2cinemagraph/text2cinemagraph","repo_kind":"official","path":"models/normalization.py","file_url":"https://github.com/text2cinemagraph/text2cinemagraph/blob/HEAD/models/normalization.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ef64c9bacc1b00b0"}},{"code_sha256_prefix":"066d51ac5067e804","entry":"get_linear_layer","repo":"text2cinemagraph/text2cinemagraph","repo_kind":"official","path":"models/normalization.py","file_url":"https://github.com/text2cinemagraph/text2cinemagraph/blob/HEAD/models/normalization.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"066d51ac5067e804"}},{"code_sha256_prefix":"29bb0088c8c38ae4","entry":"get_norm_layer","repo":"text2cinemagraph/text2cinemagraph","repo_kind":"official","path":"models/networks.py","file_url":"https://github.com/text2cinemagraph/text2cinemagraph/blob/HEAD/models/networks.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"29bb0088c8c38ae4"}},{"code_sha256_prefix":"2346c2f6fbbc68bf","entry":"get_resnet_arch","repo":"text2cinemagraph/text2cinemagraph","repo_kind":"official","path":"models/configs.py","file_url":"https://github.com/text2cinemagraph/text2cinemagraph/blob/HEAD/models/configs.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2346c2f6fbbc68bf"}},{"code_sha256_prefix":"81e8721e2bf7575c","entry":"lcm","repo":"text2cinemagraph/text2cinemagraph","repo_kind":"official","path":"train_motion.py","file_url":"https://github.com/text2cinemagraph/text2cinemagraph/blob/HEAD/train_motion.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"81e8721e2bf7575c"}},{"code_sha256_prefix":"eb055361deea8a74","entry":"modify_sd2pix2pixHD_keys","repo":"text2cinemagraph/text2cinemagraph","repo_kind":"official","path":"models/networks.py","file_url":"https://github.com/text2cinemagraph/text2cinemagraph/blob/HEAD/models/networks.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eb055361deea8a74"}},{"code_sha256_prefix":"9a299fe5ae09e407","entry":"uniq","repo":"text2cinemagraph/text2cinemagraph","repo_kind":"official","path":"models/attention.py","file_url":"https://github.com/text2cinemagraph/text2cinemagraph/blob/HEAD/models/attention.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9a299fe5ae09e407"}},{"code_sha256_prefix":"34fd565f2856a69f","entry":"create_model","repo":"text2cinemagraph/text2cinemagraph","repo_kind":"official","path":"models/models.py","file_url":"https://github.com/text2cinemagraph/text2cinemagraph/blob/HEAD/models/models.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"34fd565f2856a69f"}},{"code_sha256_prefix":"4ceac51c7b8e4270","entry":"cupy_kernel","repo":"text2cinemagraph/text2cinemagraph","repo_kind":"official","path":"models/softsplat.py","file_url":"https://github.com/text2cinemagraph/text2cinemagraph/blob/HEAD/models/softsplat.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4ceac51c7b8e4270"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}