{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/decoding-natural-images-from-eeg-for-object","title":"Decoding Natural Images from EEG for Object Recognition","arxiv_id":"2308.13234","date":"2023-08-25","proceeding":null,"authors":["Yonghao Song","Bingchuan Liu","Xiang Li","Nanlin Shi","Yijun Wang","Xiaorong Gao"],"abstract":"Electroencephalography (EEG) signals, known for convenient non-invasive acquisition but low signal-to-noise ratio, have recently gained substantial attention due to the potential to decode natural images. This paper presents a self-supervised framework to demonstrate the feasibility of learning image representations from EEG signals, particularly for object recognition. The framework utilizes image and EEG encoders to extract features from paired image stimuli and EEG responses. Contrastive learning aligns these two modalities by constraining their similarity. With the framework, we attain significantly above-chance results on a comprehensive EEG-image dataset, achieving a top-1 accuracy of 15.6% and a top-5 accuracy of 42.8% in challenging 200-way zero-shot tasks. Moreover, we perform extensive experiments to explore the biological plausibility by resolving the temporal, spatial, spectral, and semantic aspects of EEG signals. Besides, we introduce attention modules to capture spatial correlations, providing implicit evidence of the brain activity perceived from EEG data. These findings yield valuable insights for neural decoding and brain-computer interfaces in real-world scenarios. The code will be released on https://github.com/eeyhsong/NICE-EEG.","url_abs":"https://arxiv.org/abs/2308.13234v3","url_pdf":"https://arxiv.org/pdf/2308.13234v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"decoding-natural-images-from-eeg-for-object","repo_url":"https://github.com/eeyhsong/nice-eeg","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}},{"paper_slug":"decoding-natural-images-from-eeg-for-object","repo_url":"https://github.com/dongyangli-del/eeg_image_decode","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"decoding-natural-images-from-eeg-for-object","repo_url":"https://github.com/ncclab-sustech/eeg_image_decode","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":null},{"paper_slug":"decoding-natural-images-from-eeg-for-object","repo_url":"https://github.com/xiaozhangyes/cognitioncapturer","is_official":0,"mentioned_in_paper":0,"mentioned_in_github":1,"framework":"pytorch","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":"eeg-1","task_name":"EEG"},{"task_slug":"eeg","task_name":"Electroencephalogram (EEG)"},{"task_slug":"object","task_name":"Object"},{"task_slug":"object-recognition","task_name":"Object Recognition"}],"methods":[{"method_slug":"align","method_name":"ALIGN"},{"method_slug":"contrastive-learning","method_name":"Contrastive Learning"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2308.13234","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2308.13234"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/eeyhsong/NICE-EEG","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/xiaozhangyes/cognitioncapturer","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/dongyangli-del/eeg_image_decode","reach":null},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/eeyhsong/nice-eeg","reach":{"status":"ok","spdx":"MIT"}},{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/ncclab-sustech/eeg_image_decode","reach":null}],"summary":{"ran":6,"ran_fixture":1,"unverified":5},"by_repo_kind":{"official":{"samples":8,"ran":6,"repositories":1},"listed":{"samples":4,"ran":1,"repositories":2}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"62e5aecd02415dcd","entry":"Enc_eeg","repo":"eeyhsong/nice-eeg","repo_kind":"official","path":"nice_stand.py","file_url":"https://github.com/eeyhsong/nice-eeg/blob/HEAD/nice_stand.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"62e5aecd02415dcd"}},{"code_sha256_prefix":"9aca28b9d0e02be2","entry":"FlattenHead","repo":"eeyhsong/nice-eeg","repo_kind":"official","path":"nice_stand.py","file_url":"https://github.com/eeyhsong/nice-eeg/blob/HEAD/nice_stand.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"well_formed","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"9aca28b9d0e02be2"}},{"code_sha256_prefix":"8e7c1b0649fa0c51","entry":"PatchEmbedding","repo":"eeyhsong/nice-eeg","repo_kind":"official","path":"nice_stand.py","file_url":"https://github.com/eeyhsong/nice-eeg/blob/HEAD/nice_stand.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8e7c1b0649fa0c51"}},{"code_sha256_prefix":"76f1ee846c636698","entry":"Proj_eeg","repo":"eeyhsong/nice-eeg","repo_kind":"official","path":"nice_stand.py","file_url":"https://github.com/eeyhsong/nice-eeg/blob/HEAD/nice_stand.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"76f1ee846c636698"}},{"code_sha256_prefix":"3a67493f23f496fc","entry":"Proj_img","repo":"eeyhsong/nice-eeg","repo_kind":"official","path":"nice_stand.py","file_url":"https://github.com/eeyhsong/nice-eeg/blob/HEAD/nice_stand.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":"deterministic","behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"3a67493f23f496fc"}},{"code_sha256_prefix":"8e91610149da4e52","entry":"ResidualAdd","repo":"eeyhsong/nice-eeg","repo_kind":"official","path":"nice_stand.py","file_url":"https://github.com/eeyhsong/nice-eeg/blob/HEAD/nice_stand.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":true,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8e91610149da4e52"}},{"code_sha256_prefix":"c7f85279d772ea19","entry":"itm_eval","repo":"xiaozhangyes/cognitioncapturer","repo_kind":"listed","path":"src/models/components/utils.py","file_url":"https://github.com/xiaozhangyes/cognitioncapturer/blob/HEAD/src/models/components/utils.py","link_basis":"harvester_set","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c7f85279d772ea19"}},{"code_sha256_prefix":"4515ed9ed335ba90","entry":"DiffusionPrior","repo":"ncclab-sustech/eeg_image_decode","repo_kind":"listed","path":"Generation/diffusion_prior.py","file_url":"https://github.com/ncclab-sustech/eeg_image_decode/blob/HEAD/Generation/diffusion_prior.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"4515ed9ed335ba90"}},{"code_sha256_prefix":"8b7cf593d55de63c","entry":"IE","repo":"eeyhsong/nice-eeg","repo_kind":"official","path":"nice_stand.py","file_url":"https://github.com/eeyhsong/nice-eeg/blob/HEAD/nice_stand.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"8b7cf593d55de63c"}},{"code_sha256_prefix":"e2c3c0b6fb6fe418","entry":"evaluation","repo":"xiaozhangyes/cognitioncapturer","repo_kind":"listed","path":"src/models/components/utils.py","file_url":"https://github.com/xiaozhangyes/cognitioncapturer/blob/HEAD/src/models/components/utils.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"e2c3c0b6fb6fe418"}},{"code_sha256_prefix":"bf26f5df6af6c707","entry":"get_model_structure","repo":"xiaozhangyes/cognitioncapturer","repo_kind":"listed","path":"src/models/components/Cogcap/Cogcap.py","file_url":"https://github.com/xiaozhangyes/cognitioncapturer/blob/HEAD/src/models/components/Cogcap/Cogcap.py","link_basis":"harvester_set","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"bf26f5df6af6c707"}},{"code_sha256_prefix":"fecc771b6daaf666","entry":"weights_init_normal","repo":"eeyhsong/nice-eeg","repo_kind":"official","path":"nice_stand.py","file_url":"https://github.com/eeyhsong/nice-eeg/blob/HEAD/nice_stand.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"fecc771b6daaf666"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}