{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/dragvideo-interactive-drag-style-video","title":"DragVideo: Interactive Drag-style Video Editing","arxiv_id":"2312.02216","date":"2023-12-03","proceeding":null,"authors":["Yufan Deng","Ruida Wang","Yuhao Zhang","Yu-Wing Tai","Chi-Keung Tang"],"abstract":"Video generation models have shown their superior ability to generate photo-realistic video. However, how to accurately control (or edit) the video remains a formidable challenge. The main issues are: 1) how to perform direct and accurate user control in editing; 2) how to execute editings like changing shape, expression, and layout without unsightly distortion and artifacts to the edited content; and 3) how to maintain spatio-temporal consistency of video after editing. To address the above issues, we propose DragVideo, a general drag-style video editing framework. Inspired by DragGAN, DragVideo addresses issues 1) and 2) by proposing the drag-style video latent optimization method which gives desired control by updating noisy video latent according to drag instructions through video-level drag objective function. We amend issue 3) by integrating the video diffusion model with sample-specific LoRA and Mutual Self-Attention in DragVideo to ensure the edited result is spatio-temporally consistent. We also present a series of testing examples for drag-style video editing and conduct extensive experiments across a wide array of challenging editing tasks, such as motion, skeleton editing, etc, underscoring DragVideo can edit video in an intuitive, faithful to the user's intention manner, with nearly unnoticeable distortion and artifacts, while maintaining spatio-temporal consistency. While traditional prompt-based video editing fails to do the former two and directly applying image drag editing fails in the last, DragVideo's versatility and generality are emphasized. Github link: https://github.com/RickySkywalker/DragVideo-Official.","url_abs":"https://arxiv.org/abs/2312.02216v3","url_pdf":"https://arxiv.org/pdf/2312.02216v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"dragvideo-interactive-drag-style-video","repo_url":"https://github.com/rickyskywalker/dragvideo-official","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"none","reach":{"status":"ok"}}],"tasks":[{"task_slug":"video-editing","task_name":"Video Editing"},{"task_slug":"video-generation","task_name":"Video Generation"}],"methods":[{"method_slug":"concatenated-skip-connection","method_name":"Concatenated Skip Connection"},{"method_slug":"convolution","method_name":"Convolution"},{"method_slug":"diffusion","method_name":"Diffusion"},{"method_slug":"max-pooling","method_name":"Max Pooling"},{"method_slug":"relu","method_name":"ReLU"},{"method_slug":"u-net","method_name":"U-Net"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":"https://syntology.ai/paper/2312.02216","atlas_url":"https://app.syntology.ai/?focus=2312.02216","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2312.02216"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-25T09:33:49+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/rickyskywalker/dragvideo-official","reach":{"status":"ok"}}],"summary":{"ran":11,"unverified":3},"by_repo_kind":{"official":{"samples":14,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":14,"samples":[{"code_sha256_prefix":"7520da07d06562e9","entry":"check_handle_reach_target","repo":"rickyskywalker/dragvideo-official","repo_kind":"official","path":"DragVideo/utils/drag_utils.py","file_url":"https://github.com/rickyskywalker/dragvideo-official/blob/HEAD/DragVideo/utils/drag_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"7520da07d06562e9"}},{"code_sha256_prefix":"2163d3aecf3aed8b","entry":"clip_point","repo":"rickyskywalker/dragvideo-official","repo_kind":"official","path":"DragVideo/utils/track_utils.py","file_url":"https://github.com/rickyskywalker/dragvideo-official/blob/HEAD/DragVideo/utils/track_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2163d3aecf3aed8b"}},{"code_sha256_prefix":"b8e40a9f892d2d03","entry":"encode_prompts","repo":"rickyskywalker/dragvideo-official","repo_kind":"official","path":"DragVideo/utils/function_utils.py","file_url":"https://github.com/rickyskywalker/dragvideo-official/blob/HEAD/DragVideo/utils/function_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b8e40a9f892d2d03"}},{"code_sha256_prefix":"2cc475a39a00b385","entry":"interpolate_feature_patch","repo":"rickyskywalker/dragvideo-official","repo_kind":"official","path":"DragVideo/utils/drag_utils.py","file_url":"https://github.com/rickyskywalker/dragvideo-official/blob/HEAD/DragVideo/utils/drag_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"2cc475a39a00b385"}},{"code_sha256_prefix":"c868d3bf65ca0b82","entry":"mask_image","repo":"rickyskywalker/dragvideo-official","repo_kind":"official","path":"DragVideo/utils/ui_utils.py","file_url":"https://github.com/rickyskywalker/dragvideo-official/blob/HEAD/DragVideo/utils/ui_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c868d3bf65ca0b82"}},{"code_sha256_prefix":"5946ab6afcea0492","entry":"override_attn_proc_forward","repo":"rickyskywalker/dragvideo-official","repo_kind":"official","path":"DragVideo/utils/attn_utils.py","file_url":"https://github.com/rickyskywalker/dragvideo-official/blob/HEAD/DragVideo/utils/attn_utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"5946ab6afcea0492"}},{"code_sha256_prefix":"d3e2787899ebd0a5","entry":"override_lora_attn_proc_forward","repo":"rickyskywalker/dragvideo-official","repo_kind":"official","path":"DragVideo/utils/attn_utils.py","file_url":"https://github.com/rickyskywalker/dragvideo-official/blob/HEAD/DragVideo/utils/attn_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d3e2787899ebd0a5"}},{"code_sha256_prefix":"d47acef028bf4d5a","entry":"point_tracking","repo":"rickyskywalker/dragvideo-official","repo_kind":"official","path":"DragVideo/utils/drag_utils.py","file_url":"https://github.com/rickyskywalker/dragvideo-official/blob/HEAD/DragVideo/utils/drag_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"d47acef028bf4d5a"}},{"code_sha256_prefix":"8262b7927af0eae8","entry":"preprocess_video","repo":"rickyskywalker/dragvideo-official","repo_kind":"official","path":"DragVideo/utils/function_utils.py","file_url":"https://github.com/rickyskywalker/dragvideo-official/blob/HEAD/DragVideo/utils/function_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"8262b7927af0eae8"}},{"code_sha256_prefix":"488e12f3404ec736","entry":"process_points_to_sup_res","repo":"rickyskywalker/dragvideo-official","repo_kind":"official","path":"DragVideo/utils/point_utils.py","file_url":"https://github.com/rickyskywalker/dragvideo-official/blob/HEAD/DragVideo/utils/point_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"488e12f3404ec736"}},{"code_sha256_prefix":"a7ba6e53579990f0","entry":"reconstruct_points_from_sup_res","repo":"rickyskywalker/dragvideo-official","repo_kind":"official","path":"DragVideo/utils/point_utils.py","file_url":"https://github.com/rickyskywalker/dragvideo-official/blob/HEAD/DragVideo/utils/point_utils.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a7ba6e53579990f0"}},{"code_sha256_prefix":"59c311004be1c0a8","entry":"encode_video","repo":"rickyskywalker/dragvideo-official","repo_kind":"official","path":"DragVideo/utils/function_utils.py","file_url":"https://github.com/rickyskywalker/dragvideo-official/blob/HEAD/DragVideo/utils/function_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"59c311004be1c0a8"}},{"code_sha256_prefix":"121fe72839a66425","entry":"register_attention_editor_diffusers","repo":"rickyskywalker/dragvideo-official","repo_kind":"official","path":"DragVideo/utils/attn_utils.py","file_url":"https://github.com/rickyskywalker/dragvideo-official/blob/HEAD/DragVideo/utils/attn_utils.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"121fe72839a66425"}},{"code_sha256_prefix":"9383dad3ca235223","entry":"text_encoder_attn_modules","repo":"rickyskywalker/dragvideo-official","repo_kind":"official","path":"DragVideo/LoRA_helper/loaders.py","file_url":"https://github.com/rickyskywalker/dragvideo-official/blob/HEAD/DragVideo/LoRA_helper/loaders.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"9383dad3ca235223"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}