{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/videomarkbench-benchmarking-robustness-of","title":"VideoMarkBench: Benchmarking Robustness of Video Watermarking","arxiv_id":"2505.21620","date":"2025-05-27","proceeding":null,"authors":["Zhengyuan Jiang","Moyang Guo","Kecen Li","Yuepeng Hu","Yupu Wang","Zhicong Huang","Cheng Hong","Neil Zhenqiang Gong"],"abstract":"The rapid development of video generative models has led to a surge in highly realistic synthetic videos, raising ethical concerns related to disinformation and copyright infringement. Recently, video watermarking has been proposed as a mitigation strategy by embedding invisible marks into AI-generated videos to enable subsequent detection. However, the robustness of existing video watermarking methods against both common and adversarial perturbations remains underexplored. In this work, we introduce VideoMarkBench, the first systematic benchmark designed to evaluate the robustness of video watermarks under watermark removal and watermark forgery attacks. Our study encompasses a unified dataset generated by three state-of-the-art video generative models, across three video styles, incorporating four watermarking methods and seven aggregation strategies used during detection. We comprehensively evaluate 12 types of perturbations under white-box, black-box, and no-box threat models. Our findings reveal significant vulnerabilities in current watermarking approaches and highlight the urgent need for more robust solutions. Our code is available at https://github.com/zhengyuan-jiang/VideoMarkBench.","url_abs":"https://arxiv.org/abs/2505.21620v1","url_pdf":"https://arxiv.org/pdf/2505.21620v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"videomarkbench-benchmarking-robustness-of","repo_url":"https://github.com/zhengyuan-jiang/videomarkbench","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"benchmarking","task_name":"Benchmarking"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2505.21620","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2505.21620"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/zhengyuan-jiang/videomarkbench","reach":null}],"summary":{"ran_fixture":4},"by_repo_kind":{"official":{"samples":4,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"a1b7bb0950864b5d","entry":"bit_majority","repo":"zhengyuan-jiang/videomarkbench","repo_kind":"official","path":"aggregation.py","file_url":"https://github.com/zhengyuan-jiang/videomarkbench/blob/HEAD/aggregation.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a1b7bb0950864b5d"}},{"code_sha256_prefix":"f14d3c940f9e040e","entry":"framenorm","repo":"zhengyuan-jiang/videomarkbench","repo_kind":"official","path":"blackbox_perturbations/square_attack/adaptations/REVMark.py","file_url":"https://github.com/zhengyuan-jiang/videomarkbench/blob/HEAD/blackbox_perturbations/square_attack/adaptations/REVMark.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f14d3c940f9e040e"}},{"code_sha256_prefix":"d104660de15a7eda","entry":"logit_g_median","repo":"zhengyuan-jiang/videomarkbench","repo_kind":"official","path":"aggregation.py","file_url":"https://github.com/zhengyuan-jiang/videomarkbench/blob/HEAD/aggregation.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"d104660de15a7eda"}},{"code_sha256_prefix":"51ab74fed57299b6","entry":"logit_mean","repo":"zhengyuan-jiang/videomarkbench","repo_kind":"official","path":"aggregation.py","file_url":"https://github.com/zhengyuan-jiang/videomarkbench/blob/HEAD/aggregation.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"51ab74fed57299b6"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}