{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/eventclip-adapting-clip-for-event-based","title":"EventCLIP: Adapting CLIP for Event-based Object Recognition","arxiv_id":"2306.06354","date":"2023-06-10","proceeding":null,"authors":["Ziyi Wu","Xudong Liu","Igor Gilitschenski"],"abstract":"Recent advances in zero-shot and few-shot classification heavily rely on the success of pre-trained vision-language models (VLMs) such as CLIP. Due to a shortage of large-scale datasets, training such models for event camera data remains infeasible. Thus, adapting existing VLMs across modalities to event vision is an important research challenge. In this work, we introduce EventCLIP, a novel approach that utilizes CLIP for zero-shot and few-shot event-based object recognition. We first generalize CLIP's image encoder to event data by converting raw events to 2D grid-based representations. To further enhance performance, we propose a feature adapter to aggregate temporal information over event frames and refine text embeddings to better align with the visual inputs. We evaluate EventCLIP on N-Caltech, N-Cars, and N-ImageNet datasets, achieving state-of-the-art few-shot performance. When fine-tuned on the entire dataset, our method outperforms all existing event classifiers. Moreover, we explore practical applications of EventCLIP including robust event classification and label-free event recognition, where our approach surpasses previous baselines designed specifically for these tasks.","url_abs":"https://arxiv.org/abs/2306.06354v3","url_pdf":"https://arxiv.org/pdf/2306.06354v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"eventclip-adapting-clip-for-event-based","repo_url":"https://github.com/Wuziyi616/EventCLIP","is_official":1,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"few-shot-learning","task_name":"Few-Shot Learning"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-recognition","task_name":"Object Recognition"},{"task_slug":"pseudo-label","task_name":"Pseudo Label"},{"task_slug":"robust-classification","task_name":"Robust classification"}],"methods":[{"method_slug":"adapter","method_name":"Adapter"},{"method_slug":"clip","method_name":"CLIP"},{"method_slug":"self-learning","method_name":"Self-Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2306.06354","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2306.06354"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/Wuziyi616/EventCLIP","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"unverified":10},"by_repo_kind":{"official":{"samples":10,"ran":0,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"8aff5a54678b2041","entry":"build_event2img_dataset","repo":"Wuziyi616/EventCLIP","repo_kind":"official","path":"datasets/event2img.py","file_url":"https://github.com/Wuziyi616/EventCLIP/blob/HEAD/datasets/event2img.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8aff5a54678b2041"}},{"code_sha256_prefix":"ced40880df80f9a6","entry":"build_lora_merged_proj","repo":"Wuziyi616/EventCLIP","repo_kind":"official","path":"models/lora.py","file_url":"https://github.com/Wuziyi616/EventCLIP/blob/HEAD/models/lora.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"ced40880df80f9a6"}},{"code_sha256_prefix":"dac43eb982742334","entry":"build_lora_mha","repo":"Wuziyi616/EventCLIP","repo_kind":"official","path":"models/lora.py","file_url":"https://github.com/Wuziyi616/EventCLIP/blob/HEAD/models/lora.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"dac43eb982742334"}},{"code_sha256_prefix":"8db8291e6f0c223c","entry":"build_lora_proj","repo":"Wuziyi616/EventCLIP","repo_kind":"official","path":"models/lora.py","file_url":"https://github.com/Wuziyi616/EventCLIP/blob/HEAD/models/lora.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8db8291e6f0c223c"}},{"code_sha256_prefix":"8f7ff725c27a34c8","entry":"denormalize","repo":"Wuziyi616/EventCLIP","repo_kind":"official","path":"method.py","file_url":"https://github.com/Wuziyi616/EventCLIP/blob/HEAD/method.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8f7ff725c27a34c8"}},{"code_sha256_prefix":"8f09a1a21e5e373e","entry":"find_key_from_value","repo":"Wuziyi616/EventCLIP","repo_kind":"official","path":"gen_data.py","file_url":"https://github.com/Wuziyi616/EventCLIP/blob/HEAD/gen_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8f09a1a21e5e373e"}},{"code_sha256_prefix":"58255d56af8ce2ce","entry":"get_folder_and_fn","repo":"Wuziyi616/EventCLIP","repo_kind":"official","path":"gen_data.py","file_url":"https://github.com/Wuziyi616/EventCLIP/blob/HEAD/gen_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"58255d56af8ce2ce"}},{"code_sha256_prefix":"c3e86ebaae72bca0","entry":"get_real_path","repo":"Wuziyi616/EventCLIP","repo_kind":"official","path":"gen_data.py","file_url":"https://github.com/Wuziyi616/EventCLIP/blob/HEAD/gen_data.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c3e86ebaae72bca0"}},{"code_sha256_prefix":"89cdfb6c9efa1bae","entry":"get_real_path","repo":"Wuziyi616/EventCLIP","repo_kind":"official","path":"datasets/caltech.py","file_url":"https://github.com/Wuziyi616/EventCLIP/blob/HEAD/datasets/caltech.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"89cdfb6c9efa1bae"}},{"code_sha256_prefix":"a3370a1b6b0ac33c","entry":"load_event","repo":"Wuziyi616/EventCLIP","repo_kind":"official","path":"datasets/imagenet.py","file_url":"https://github.com/Wuziyi616/EventCLIP/blob/HEAD/datasets/imagenet.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"a3370a1b6b0ac33c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}