{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/follow-your-click-open-domain-regional-image","title":"Follow-Your-Click: Open-domain Regional Image Animation via Short Prompts","arxiv_id":"2403.08268","date":"2024-03-13","proceeding":null,"authors":["Yue Ma","Yingqing He","Hongfa Wang","Andong Wang","Chenyang Qi","Chengfei Cai","Xiu Li","Zhifeng Li","Heung-Yeung Shum","Wei Liu","Qifeng Chen"],"abstract":"Despite recent advances in image-to-video generation, better controllability and local animation are less explored. Most existing image-to-video methods are not locally aware and tend to move the entire scene. However, human artists may need to control the movement of different objects or regions. Additionally, current I2V methods require users not only to describe the target motion but also to provide redundant detailed descriptions of frame contents. These two issues hinder the practical utilization of current I2V tools. In this paper, we propose a practical framework, named Follow-Your-Click, to achieve image animation with a simple user click (for specifying what to move) and a short motion prompt (for specifying how to move). Technically, we propose the first-frame masking strategy, which significantly improves the video generation quality, and a motion-augmented module equipped with a short motion prompt dataset to improve the short prompt following abilities of our model. To further control the motion speed, we propose flow-based motion magnitude control to control the speed of target movement more precisely. Our framework has simpler yet precise user control and better generation performance than previous methods. Extensive experiments compared with 7 baselines, including both commercial tools and research methods on 8 metrics, suggest the superiority of our approach. Project Page: https://follow-your-click.github.io/","url_abs":"https://arxiv.org/abs/2403.08268v1","url_pdf":"https://arxiv.org/pdf/2403.08268v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"follow-your-click-open-domain-regional-image","repo_url":"https://github.com/mayuelala/followyourclick","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"image-animation","task_name":"Image Animation"},{"task_slug":"image-to-video","task_name":"Image to Video Generation"},{"task_slug":"video-generation","task_name":"Video Generation"}],"methods":[{"method_slug":"aware","method_name":"AWARE"},{"method_slug":"speed","method_name":"SPEED"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2403.08268","atlas_url":"https://app.syntology.ai/?focus=2403.08268","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2403.08268"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/mayuelala/followyourclick","reach":{"status":"ok"}}],"summary":{"ran":5,"unverified":4},"by_repo_kind":{"official":{"samples":9,"ran":5,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":9,"samples":[{"code_sha256_prefix":"7d657d43bf8f10fc","entry":"convert_pytorch_state_dict_to_flax","repo":"mayuelala/followyourclick","repo_kind":"official","path":"diffusers/modeling_flax_pytorch_utils.py","file_url":"https://github.com/mayuelala/followyourclick/blob/HEAD/diffusers/modeling_flax_pytorch_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7d657d43bf8f10fc"}},{"code_sha256_prefix":"eafa252fd6f86857","entry":"get_parameter_device","repo":"mayuelala/followyourclick","repo_kind":"official","path":"diffusers/modeling_utils.py","file_url":"https://github.com/mayuelala/followyourclick/blob/HEAD/diffusers/modeling_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"eafa252fd6f86857"}},{"code_sha256_prefix":"dc2045840ce679cb","entry":"get_parameter_dtype","repo":"mayuelala/followyourclick","repo_kind":"official","path":"diffusers/modeling_utils.py","file_url":"https://github.com/mayuelala/followyourclick/blob/HEAD/diffusers/modeling_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"dc2045840ce679cb"}},{"code_sha256_prefix":"fa58367fb507f72f","entry":"rename_key","repo":"mayuelala/followyourclick","repo_kind":"official","path":"diffusers/modeling_flax_pytorch_utils.py","file_url":"https://github.com/mayuelala/followyourclick/blob/HEAD/diffusers/modeling_flax_pytorch_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"fa58367fb507f72f"}},{"code_sha256_prefix":"5f1d8a0f6c5bcb3e","entry":"rename_key_and_reshape_tensor","repo":"mayuelala/followyourclick","repo_kind":"official","path":"diffusers/modeling_flax_pytorch_utils.py","file_url":"https://github.com/mayuelala/followyourclick/blob/HEAD/diffusers/modeling_flax_pytorch_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5f1d8a0f6c5bcb3e"}},{"code_sha256_prefix":"301682fc3342590a","entry":"check_imports","repo":"mayuelala/followyourclick","repo_kind":"official","path":"diffusers/dynamic_modules_utils.py","file_url":"https://github.com/mayuelala/followyourclick/blob/HEAD/diffusers/dynamic_modules_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"301682fc3342590a"}},{"code_sha256_prefix":"00cc853a73f3aeb6","entry":"get_relative_import_files","repo":"mayuelala/followyourclick","repo_kind":"official","path":"diffusers/dynamic_modules_utils.py","file_url":"https://github.com/mayuelala/followyourclick/blob/HEAD/diffusers/dynamic_modules_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"00cc853a73f3aeb6"}},{"code_sha256_prefix":"450aeffe7f80f917","entry":"get_relative_imports","repo":"mayuelala/followyourclick","repo_kind":"official","path":"diffusers/dynamic_modules_utils.py","file_url":"https://github.com/mayuelala/followyourclick/blob/HEAD/diffusers/dynamic_modules_utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"450aeffe7f80f917"}},{"code_sha256_prefix":"b57cacefe65c27ce","entry":"load_state_dict","repo":"mayuelala/followyourclick","repo_kind":"official","path":"diffusers/modeling_utils.py","file_url":"https://github.com/mayuelala/followyourclick/blob/HEAD/diffusers/modeling_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b57cacefe65c27ce"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}