{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/ophnet-a-large-scale-video-benchmark-for","title":"OphNet: A Large-Scale Video Benchmark for Ophthalmic Surgical Workflow Understanding","arxiv_id":"2406.07471","date":"2024-06-11","proceeding":null,"authors":["Ming Hu","Peng Xia","Lin Wang","Siyuan Yan","Feilong Tang","Zhongxing Xu","Yimin Luo","Kaimin Song","Jurgen Leitner","Xuelian Cheng","Jun Cheng","Chi Liu","Kaijing Zhou","ZongYuan Ge"],"abstract":"Surgical scene perception via videos is critical for advancing robotic surgery, telesurgery, and AI-assisted surgery, particularly in ophthalmology. However, the scarcity of diverse and richly annotated video datasets has hindered the development of intelligent systems for surgical workflow analysis. Existing datasets face challenges such as small scale, lack of diversity in surgery and phase categories, and absence of time-localized annotations. These limitations impede action understanding and model generalization validation in complex and diverse real-world surgical scenarios. To address this gap, we introduce OphNet, a large-scale, expert-annotated video benchmark for ophthalmic surgical workflow understanding. OphNet features: 1) A diverse collection of 2,278 surgical videos spanning 66 types of cataract, glaucoma, and corneal surgeries, with detailed annotations for 102 unique surgical phases and 150 fine-grained operations. 2) Sequential and hierarchical annotations for each surgery, phase, and operation, enabling comprehensive understanding and improved interpretability. 3) Time-localized annotations, facilitating temporal localization and prediction tasks within surgical workflows. With approximately 285 hours of surgical videos, OphNet is about 20 times larger than the largest existing surgical workflow analysis benchmark. Code and dataset are available at: https://minghu0830.github.io/OphNet-benchmark/.","url_abs":"https://arxiv.org/abs/2406.07471v4","url_pdf":"https://arxiv.org/pdf/2406.07471v4.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"ophnet-a-large-scale-video-benchmark-for","repo_url":"https://github.com/minghu0830/ophnet-benchmark","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"action-understanding","task_name":"Action Understanding"},{"task_slug":"diversity","task_name":"Diversity"},{"task_slug":"temporal-localization","task_name":"Temporal Localization"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.07471","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.07471"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/minghu0830/ophnet-benchmark","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran":9,"ran_honours":1,"unverified":6},"by_repo_kind":{"official":{"samples":16,"ran":10,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"bc39b8ed5fefa7eb","entry":"crop_clip","repo":"minghu0830/ophnet-benchmark","repo_kind":"official","path":"baselines/task2/backbone/videomaev2/dataset/functional.py","file_url":"https://github.com/minghu0830/ophnet-benchmark/blob/HEAD/baselines/task2/backbone/videomaev2/dataset/functional.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bc39b8ed5fefa7eb"}},{"code_sha256_prefix":"2b53e457d5632b4e","entry":"get_resize_sizes","repo":"minghu0830/ophnet-benchmark","repo_kind":"official","path":"baselines/task2/backbone/videomaev2/dataset/functional.py","file_url":"https://github.com/minghu0830/ophnet-benchmark/blob/HEAD/baselines/task2/backbone/videomaev2/dataset/functional.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"2b53e457d5632b4e"}},{"code_sha256_prefix":"da651e3979a18f84","entry":"get_sinusoid_encoding_table","repo":"minghu0830/ophnet-benchmark","repo_kind":"official","path":"baselines/task2/backbone/videomaev2/models/modeling_finetune.py","file_url":"https://github.com/minghu0830/ophnet-benchmark/blob/HEAD/baselines/task2/backbone/videomaev2/models/modeling_finetune.py","link_basis":"harvester_set","language":"python","status":"ran_honours","verification_level":1,"contract_check":"HONOURS","metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"da651e3979a18f84"}},{"code_sha256_prefix":"cf2d8113b18afcf4","entry":"process_video","repo":"minghu0830/ophnet-benchmark","repo_kind":"official","path":"data_processing/cliper.py","file_url":"https://github.com/minghu0830/ophnet-benchmark/blob/HEAD/data_processing/cliper.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cf2d8113b18afcf4"}},{"code_sha256_prefix":"8d1dc1aa52fe2112","entry":"resize","repo":"minghu0830/ophnet-benchmark","repo_kind":"official","path":"baselines/task2/backbone/videomaev2/extract_tad_feature.py","file_url":"https://github.com/minghu0830/ophnet-benchmark/blob/HEAD/baselines/task2/backbone/videomaev2/extract_tad_feature.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8d1dc1aa52fe2112"}},{"code_sha256_prefix":"981936c73d3cdd12","entry":"resize_clip","repo":"minghu0830/ophnet-benchmark","repo_kind":"official","path":"baselines/task2/backbone/videomaev2/dataset/functional.py","file_url":"https://github.com/minghu0830/ophnet-benchmark/blob/HEAD/baselines/task2/backbone/videomaev2/dataset/functional.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"981936c73d3cdd12"}},{"code_sha256_prefix":"e866e1196986f728","entry":"shear_x","repo":"minghu0830/ophnet-benchmark","repo_kind":"official","path":"baselines/task2/backbone/videomaev2/dataset/rand_augment.py","file_url":"https://github.com/minghu0830/ophnet-benchmark/blob/HEAD/baselines/task2/backbone/videomaev2/dataset/rand_augment.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e866e1196986f728"}},{"code_sha256_prefix":"7b2503c99c9447fb","entry":"shear_y","repo":"minghu0830/ophnet-benchmark","repo_kind":"official","path":"baselines/task2/backbone/videomaev2/dataset/rand_augment.py","file_url":"https://github.com/minghu0830/ophnet-benchmark/blob/HEAD/baselines/task2/backbone/videomaev2/dataset/rand_augment.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7b2503c99c9447fb"}},{"code_sha256_prefix":"eab677b96697e36b","entry":"to_normalized_float_tensor","repo":"minghu0830/ophnet-benchmark","repo_kind":"official","path":"baselines/task2/backbone/videomaev2/extract_tad_feature.py","file_url":"https://github.com/minghu0830/ophnet-benchmark/blob/HEAD/baselines/task2/backbone/videomaev2/extract_tad_feature.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eab677b96697e36b"}},{"code_sha256_prefix":"323759f906fbb394","entry":"translate_x_rel","repo":"minghu0830/ophnet-benchmark","repo_kind":"official","path":"baselines/task2/backbone/videomaev2/dataset/rand_augment.py","file_url":"https://github.com/minghu0830/ophnet-benchmark/blob/HEAD/baselines/task2/backbone/videomaev2/dataset/rand_augment.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"323759f906fbb394"}},{"code_sha256_prefix":"283de3b7dcb770a3","entry":"build_dataset","repo":"minghu0830/ophnet-benchmark","repo_kind":"official","path":"baselines/task2/backbone/videomaev2/dataset/build.py","file_url":"https://github.com/minghu0830/ophnet-benchmark/blob/HEAD/baselines/task2/backbone/videomaev2/dataset/build.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"283de3b7dcb770a3"}},{"code_sha256_prefix":"cb5e3e409ec627d3","entry":"build_pretraining_dataset","repo":"minghu0830/ophnet-benchmark","repo_kind":"official","path":"baselines/task2/backbone/videomaev2/dataset/build.py","file_url":"https://github.com/minghu0830/ophnet-benchmark/blob/HEAD/baselines/task2/backbone/videomaev2/dataset/build.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"cb5e3e409ec627d3"}},{"code_sha256_prefix":"b41eabb5a3021ac4","entry":"get_image_loader","repo":"minghu0830/ophnet-benchmark","repo_kind":"official","path":"baselines/task2/backbone/videomaev2/dataset/loader.py","file_url":"https://github.com/minghu0830/ophnet-benchmark/blob/HEAD/baselines/task2/backbone/videomaev2/dataset/loader.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"b41eabb5a3021ac4"}},{"code_sha256_prefix":"0ad207d6f1eb55ed","entry":"load_backbone_model","repo":"minghu0830/ophnet-benchmark","repo_kind":"official","path":"baselines/task2/backbone/videomaev2/inference.py","file_url":"https://github.com/minghu0830/ophnet-benchmark/blob/HEAD/baselines/task2/backbone/videomaev2/inference.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0ad207d6f1eb55ed"}},{"code_sha256_prefix":"7e49117a60f3b108","entry":"spatial_sampling","repo":"minghu0830/ophnet-benchmark","repo_kind":"official","path":"baselines/task2/backbone/videomaev2/dataset/datasets.py","file_url":"https://github.com/minghu0830/ophnet-benchmark/blob/HEAD/baselines/task2/backbone/videomaev2/dataset/datasets.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"7e49117a60f3b108"}},{"code_sha256_prefix":"14b3b13b2e7ee844","entry":"tensor_normalize","repo":"minghu0830/ophnet-benchmark","repo_kind":"official","path":"baselines/task2/backbone/videomaev2/dataset/datasets.py","file_url":"https://github.com/minghu0830/ophnet-benchmark/blob/HEAD/baselines/task2/backbone/videomaev2/dataset/datasets.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"14b3b13b2e7ee844"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}