{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/common-and-rare-fundus-diseases","title":"Enhancing Diagnostic Accuracy in Rare and Common Fundus Diseases with a Knowledge-Rich Vision-Language Model","arxiv_id":"2406.09317","date":"2024-06-13","proceeding":null,"authors":["Meng Wang","Tian Lin","Aidi Lin","Kai Yu","Yuanyuan Peng","Lianyu Wang","Cheng Chen","Ke Zou","Huiyu Liang","Man Chen","Xue Yao","Meiqin Zhang","Binwei Huang","Chaoxin Zheng","Peixin Zhang","Wei Chen","Yilong Luo","Yifan Chen","Honghe Xia","Tingkun Shi","Qi Zhang","Jinming Guo","Xiaolin Chen","Jingcheng Wang","Yih Chung Tham","Dianbo Liu","Wendy Wong","Sahil Thakur","Beau Fenner","Danqi Fang","Siying Liu","Qingyun Liu","Yuqiang Huang","Hongqiang Zeng","Yanda Meng","Yukun Zhou","Zehua Jiang","Minghui Qiu","Changqing Zhang","Xinjian Chen","Sophia Y. Wang","Cecilia S. Lee","Lucia Sobrin","Carol Y Cheung","Chi Pui Pang","Pearse A. Keane","Ching-Yu Cheng","Haoyu Chen","Huazhu Fu"],"abstract":"Previous foundation models for fundus images were pre-trained with limited disease categories and knowledge base. Here we introduce a knowledge-rich vision-language model (RetiZero) that leverages knowledge from more than 400 fundus diseases. For RetiZero's pretraining, we compiled 341,896 fundus images paired with texts, sourced from public datasets, ophthalmic literature, and online resources, encompassing a diverse range of diseases across multiple ethnicities and countries. RetiZero exhibits remarkable performance in several downstream tasks, including zero-shot disease recognition, image-to-image retrieval, AI-assisted clinical diagnosis,few-shot fine-tuning, and internal- and cross-domain disease identification. In zero-shot scenarios, RetiZero achieves Top-5 accuracies of 0.843 for 15 diseases and 0.756 for 52 diseases. For image retrieval, it achieves Top-5 scores of 0.950 and 0.886 for the same sets, respectively. AI-assisted clinical diagnosis results show that RetiZero's Top-3 zero-shot performance surpasses the average of 19 ophthalmologists from Singapore, China, and the United States. RetiZero substantially enhances clinicians' accuracy in diagnosing fundus diseases, in particularly rare ones. These findings underscore the value of integrating the RetiZero into clinical settings, where various fundus diseases are encountered.","url_abs":"https://arxiv.org/abs/2406.09317v3","url_pdf":"https://arxiv.org/pdf/2406.09317v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"common-and-rare-fundus-diseases","repo_url":"https://github.com/LooKing9218/RetiZero","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":0,"framework":"pytorch","reach":null}],"tasks":[{"task_slug":"diagnostic","task_name":"Diagnostic"},{"task_slug":"image-retrieval","task_name":"Image Retrieval"},{"task_slug":"language-modeling","task_name":"Language Modeling"},{"task_slug":"language-modelling","task_name":"Language Modelling"},{"task_slug":"retrieval","task_name":"Retrieval"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2406.09317","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2406.09317"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/LooKing9218/RetiZero","reach":null}],"summary":{"ran_fixture":1},"by_repo_kind":{"official":{"samples":1,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":1,"samples":[{"code_sha256_prefix":"30240001cf3459dc","entry":"val","repo":"LooKing9218/RetiZero","repo_kind":"official","path":"Finetuning.py","file_url":"https://github.com/LooKing9218/RetiZero/blob/HEAD/Finetuning.py","link_basis":"first_harvest_node","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"30240001cf3459dc"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}