{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/phinets-brain-inspired-non-contrastive","title":"PhiNets: Brain-inspired Non-contrastive Learning Based on Temporal Prediction Hypothesis","arxiv_id":"2405.14650","date":"2024-05-23","proceeding":null,"authors":["Satoki Ishikawa","Makoto Yamada","Han Bao","Yuki Takezawa"],"abstract":"Predictive coding is a theory which hypothesises that cortex predicts sensory inputs at various levels of abstraction to minimise prediction errors. Inspired by predictive coding, Chen et al. (2024) proposed another theory, temporal prediction hypothesis, to claim that sequence memory residing in hippocampus has emerged through predicting input signals from the past sensory inputs. Specifically, they supposed that the CA3 predictor in hippocampus creates synaptic delay between input signals, which is compensated by the following CA1 predictor. Though recorded neural activities were replicated based on the temporal prediction hypothesis, its validity has not been fully explored. In this work, we aim to explore the temporal prediction hypothesis from the perspective of self-supervised learning. Specifically, we focus on non-contrastive learning, which generates two augmented views of an input image and predicts one from another. Non-contrastive learning is intimately related to the temporal prediction hypothesis because the synaptic delay is implicitly created by StopGradient. Building upon a popular non-contrastive learner, SimSiam, we propose PhiNet, an extension of SimSiam to have two predictors explicitly corresponding to the CA3 and CA1, respectively. Through studying the PhiNet model, we discover two findings. First, meaningful data representations emerge in PhiNet more stably than in SimSiam. This is initially supported by our learning dynamics analysis: PhiNet is more robust to the representational collapse. Second, PhiNet adapts more quickly to newly incoming patterns in online and continual learning scenarios. For practitioners, we additionally propose an extension called X-PhiNet integrated with a momentum encoder, excelling in continual learning. All in all, our work reveals that the temporal prediction hypothesis is a reasonable model in terms of the robustness and adaptivity.","url_abs":"https://arxiv.org/abs/2405.14650v2","url_pdf":"https://arxiv.org/pdf/2405.14650v2.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[],"tasks":[{"task_slug":"continual-learning","task_name":"Continual Learning"},{"task_slug":"contrastive-learning","task_name":"Contrastive Learning"},{"task_slug":null,"task_name":"Hippocampus"},{"task_slug":"prediction","task_name":"Prediction"},{"task_slug":"self-supervised-learning","task_name":"Self-Supervised Learning"}],"methods":[{"method_slug":"focus","method_name":"Focus"},{"method_slug":"weight-decay","method_name":"Weight Decay"}],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"syntology_url":null,"atlas_url":"https://app.syntology.ai/?focus=2405.14650","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2405.14650"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"deterministic:regex_extraction","url":"https://github.com/riverstone496/PhiNets","reach":{"status":"ok"}}],"summary":{"ran_fixture":1,"ran":3,"unverified":1},"by_repo_kind":{"found_in_text":{"samples":5,"ran":4,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":5,"samples":[{"code_sha256_prefix":"b2429807c9efa3f7","entry":"D","repo":"riverstone496/PhiNets","repo_kind":"found_in_text","path":"models/eye_phinet_mom_cos.py","file_url":"https://github.com/riverstone496/PhiNets/blob/HEAD/models/eye_phinet_mom_cos.py","link_basis":"plan_row","language":"python","status":"ran_fixture","verification_level":1,"contract_check":"RAISES","metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"b2429807c9efa3f7"}},{"code_sha256_prefix":"426551c44b24c308","entry":"D_nsg","repo":"riverstone496/PhiNets","repo_kind":"found_in_text","path":"models/eye_phinet_mom_cos.py","file_url":"https://github.com/riverstone496/PhiNets/blob/HEAD/models/eye_phinet_mom_cos.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"426551c44b24c308"}},{"code_sha256_prefix":"a1fedc7a6e612274","entry":"ema_model","repo":"riverstone496/PhiNets","repo_kind":"found_in_text","path":"models/eye_phinet_mom_cos.py","file_url":"https://github.com/riverstone496/PhiNets/blob/HEAD/models/eye_phinet_mom_cos.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"a1fedc7a6e612274"}},{"code_sha256_prefix":"c01f4316916ba8e0","entry":"get_args","repo":"riverstone496/PhiNets","repo_kind":"found_in_text","path":"arguments.py","file_url":"https://github.com/riverstone496/PhiNets/blob/HEAD/arguments.py","link_basis":"first_harvest_node","language":"python","status":"ran","verification_level":1,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"c01f4316916ba8e0"}},{"code_sha256_prefix":"82fe204726faff1a","entry":"to_pil_image","repo":"riverstone496/PhiNets","repo_kind":"found_in_text","path":"augmentations/phinet_aug.py","file_url":"https://github.com/riverstone496/PhiNets/blob/HEAD/augmentations/phinet_aug.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"NONE","inline_ok":false,"mcp_get_code":{"code_sha256":"82fe204726faff1a"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}