{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/videophy-2-a-challenging-action-centric","title":"VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generation","arxiv_id":"2503.06800","date":"2025-03-09","proceeding":null,"authors":["Hritik Bansal","Clark Peng","Yonatan Bitton","Roman Goldenberg","Aditya Grover","Kai-Wei Chang"],"abstract":"Large-scale video generative models, capable of creating realistic videos of diverse visual concepts, are strong candidates for general-purpose physical world simulators. However, their adherence to physical commonsense across real-world actions remains unclear (e.g., playing tennis, backflip). Existing benchmarks suffer from limitations such as limited size, lack of human evaluation, sim-to-real gaps, and absence of fine-grained physical rule analysis. To address this, we introduce VideoPhy-2, an action-centric dataset for evaluating physical commonsense in generated videos. We curate 200 diverse actions and detailed prompts for video synthesis from modern generative models. We perform human evaluation that assesses semantic adherence, physical commonsense, and grounding of physical rules in the generated videos. Our findings reveal major shortcomings, with even the best model achieving only 22% joint performance (i.e., high semantic and physical commonsense adherence) on the hard subset of VideoPhy-2. We find that the models particularly struggle with conservation laws like mass and momentum. Finally, we also train VideoPhy-AutoEval, an automatic evaluator for fast, reliable assessment on our dataset. Overall, VideoPhy-2 serves as a rigorous benchmark, exposing critical gaps in video generative models and guiding future research in physically-grounded video generation. The data and code is available at https://videophy2.github.io/.","url_abs":"https://arxiv.org/abs/2503.06800v1","url_pdf":"https://arxiv.org/pdf/2503.06800v1.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"videophy-2-a-challenging-action-centric","repo_url":"https://github.com/Hritikbansal/videophy/tree/main/VIDEOPHY2","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":0,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"video-generation","task_name":"Video Generation"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2503.06800","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2503.06800"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Hritikbansal/videophy/tree/main/VIDEOPHY2","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_fixture":3,"ran":5,"ran_draft_wrong":2,"ran_violates":1,"unverified":6},"by_repo_kind":{"listed":{"samples":17,"ran":11,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"78ca1debd9c4ed87","entry":"autocontrast_func","repo":"Hritikbansal/videophy","repo_kind":"listed","path":"VIDEOPHY2/data_utils/randaugment.py","file_url":"https://github.com/Hritikbansal/videophy/blob/HEAD/VIDEOPHY2/data_utils/randaugment.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"DEP_MISSING","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"78ca1debd9c4ed87"}},{"code_sha256_prefix":"45711ccd8932d38a","entry":"detokenize_generations","repo":"Hritikbansal/videophy","repo_kind":"listed","path":"VIDEOPHY2/mplug_owl_video/processing_mplug_owl.py","file_url":"https://github.com/Hritikbansal/videophy/blob/HEAD/VIDEOPHY2/mplug_owl_video/processing_mplug_owl.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"45711ccd8932d38a"}},{"code_sha256_prefix":"4c53169f63a1fcef","entry":"equalize_func","repo":"Hritikbansal/videophy","repo_kind":"listed","path":"VIDEOPHY2/data_utils/randaugment.py","file_url":"https://github.com/Hritikbansal/videophy/blob/HEAD/VIDEOPHY2/data_utils/randaugment.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"DEP_MISSING","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4c53169f63a1fcef"}},{"code_sha256_prefix":"97b29adf427aa64e","entry":"get_cosine_schedule_with_warmup","repo":"Hritikbansal/videophy","repo_kind":"listed","path":"VIDEOPHY2/utils.py","file_url":"https://github.com/Hritikbansal/videophy/blob/HEAD/VIDEOPHY2/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"97b29adf427aa64e"}},{"code_sha256_prefix":"45567e61fe09093a","entry":"get_ltor_masks_and_position_ids_from_embeddings","repo":"Hritikbansal/videophy","repo_kind":"listed","path":"VIDEOPHY2/mplug_owl_video/modeling_mplug_owl.py","file_url":"https://github.com/Hritikbansal/videophy/blob/HEAD/VIDEOPHY2/mplug_owl_video/modeling_mplug_owl.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"45567e61fe09093a"}},{"code_sha256_prefix":"265f4d55ccd60d18","entry":"get_media_indices","repo":"Hritikbansal/videophy","repo_kind":"listed","path":"VIDEOPHY2/mplug_owl_video/modeling_mplug_owl.py","file_url":"https://github.com/Hritikbansal/videophy/blob/HEAD/VIDEOPHY2/mplug_owl_video/modeling_mplug_owl.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"MISDECLARED","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"265f4d55ccd60d18"}},{"code_sha256_prefix":"752b84a6f01c43d8","entry":"get_media_types","repo":"Hritikbansal/videophy","repo_kind":"listed","path":"VIDEOPHY2/mplug_owl_video/modeling_mplug_owl.py","file_url":"https://github.com/Hritikbansal/videophy/blob/HEAD/VIDEOPHY2/mplug_owl_video/modeling_mplug_owl.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"752b84a6f01c43d8"}},{"code_sha256_prefix":"53b3ed9d6fed2b5f","entry":"get_param_groups","repo":"Hritikbansal/videophy","repo_kind":"listed","path":"VIDEOPHY2/utils.py","file_url":"https://github.com/Hritikbansal/videophy/blob/HEAD/VIDEOPHY2/utils.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"53b3ed9d6fed2b5f"}},{"code_sha256_prefix":"1dd2ab205689d45b","entry":"identity_func","repo":"Hritikbansal/videophy","repo_kind":"listed","path":"VIDEOPHY2/data_utils/randaugment.py","file_url":"https://github.com/Hritikbansal/videophy/blob/HEAD/VIDEOPHY2/data_utils/randaugment.py","link_basis":"harvester_set","language":"python","status":"ran_violates","verification_level":1,"contract_check":"VIOLATES","metamorphic_tier":"invariant","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"1dd2ab205689d45b"}},{"code_sha256_prefix":"f3a23711c5501382","entry":"load_jsonl","repo":"Hritikbansal/videophy","repo_kind":"listed","path":"VIDEOPHY2/data_utils/xgpt3_dataset.py","file_url":"https://github.com/Hritikbansal/videophy/blob/HEAD/VIDEOPHY2/data_utils/xgpt3_dataset.py","link_basis":"harvester_set","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"f3a23711c5501382"}},{"code_sha256_prefix":"6a0269eec408c1b9","entry":"modify_keys","repo":"Hritikbansal/videophy","repo_kind":"listed","path":"VIDEOPHY2/inference.py","file_url":"https://github.com/Hritikbansal/videophy/blob/HEAD/VIDEOPHY2/inference.py","link_basis":"harvester_set","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6a0269eec408c1b9"}},{"code_sha256_prefix":"8c4acc117a97109b","entry":"batchify","repo":"Hritikbansal/videophy","repo_kind":"listed","path":"VIDEOPHY2/utils.py","file_url":"https://github.com/Hritikbansal/videophy/blob/HEAD/VIDEOPHY2/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8c4acc117a97109b"}},{"code_sha256_prefix":"df4a72cfc8859ecd","entry":"build_from_cfg","repo":"Hritikbansal/videophy","repo_kind":"listed","path":"VIDEOPHY2/data_utils/registry.py","file_url":"https://github.com/Hritikbansal/videophy/blob/HEAD/VIDEOPHY2/data_utils/registry.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"df4a72cfc8859ecd"}},{"code_sha256_prefix":"a313547ee87ebfe2","entry":"deprecated_api_warning","repo":"Hritikbansal/videophy","repo_kind":"listed","path":"VIDEOPHY2/data_utils/registry.py","file_url":"https://github.com/Hritikbansal/videophy/blob/HEAD/VIDEOPHY2/data_utils/registry.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a313547ee87ebfe2"}},{"code_sha256_prefix":"65d587525000912a","entry":"get_index","repo":"Hritikbansal/videophy","repo_kind":"listed","path":"VIDEOPHY2/mplug_owl_video/processing_mplug_owl.py","file_url":"https://github.com/Hritikbansal/videophy/blob/HEAD/VIDEOPHY2/mplug_owl_video/processing_mplug_owl.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"65d587525000912a"}},{"code_sha256_prefix":"dbf18e29c76d1835","entry":"inference","repo":"Hritikbansal/videophy","repo_kind":"listed","path":"VIDEOPHY2/inference.py","file_url":"https://github.com/Hritikbansal/videophy/blob/HEAD/VIDEOPHY2/inference.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dbf18e29c76d1835"}},{"code_sha256_prefix":"a8d94a0054408e15","entry":"is_seq_of","repo":"Hritikbansal/videophy","repo_kind":"listed","path":"VIDEOPHY2/data_utils/registry.py","file_url":"https://github.com/Hritikbansal/videophy/blob/HEAD/VIDEOPHY2/data_utils/registry.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a8d94a0054408e15"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}