{"about":{"site":"https://codewithpapers.app","non_affiliation":"Code with Papers and Syntology are not affiliated with, endorsed by, or sponsored by Papers with Code, Meta, or the pwc-archive mirror.","licence":"CC BY-SA 4.0","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","attribution":"https://codewithpapers.app/attribution","modified":"archive material modified by Syntology; see the attribution page"},"url":"/paper/distributed-memory-based-self-supervised","title":"Distributed Associative Memory Network with Memory Refreshing Loss","arxiv_id":"2007.10637","date":"2020-07-21","proceeding":null,"authors":["Taewon Park","Inchul Choi","Minho Lee"],"abstract":"Despite recent progress in memory augmented neural network (MANN) research, associative memory networks with a single external memory still show limited performance on complex relational reasoning tasks. Especially the content-based addressable memory networks often fail to encode input data into rich enough representation for relational reasoning and this limits the relation modeling performance of MANN for long temporal sequence data. To address these problems, here we introduce a novel Distributed Associative Memory architecture (DAM) with Memory Refreshing Loss (MRL) which enhances the relation reasoning performance of MANN. Inspired by how the human brain works, our framework encodes data with distributed representation across multiple memory blocks and repeatedly refreshes the contents for enhanced memorization similar to the rehearsal process of the brain. For this procedure, we replace a single external memory with a set of multiple smaller associative memory blocks and update these sub-memory blocks simultaneously and independently for the distributed representation of input data. Moreover, we propose MRL which assists a task's target objective while learning relational information existing in data. MRL enables MANN to reinforce an association between input data and task objective by reproducing stochastically sampled input data from stored memory contents. With this procedure, MANN further enriches the stored representations with relational information. In experiments, we apply our approaches to Differential Neural Computer (DNC), which is one of the representative content-based addressing memory models and achieves the state-of-the-art performance on both memorization and relational reasoning tasks.","url_abs":"https://arxiv.org/abs/2007.10637v3","url_pdf":"https://arxiv.org/pdf/2007.10637v3.pdf","source":{"archive":"pwc-archive (Hugging Face), CC BY-SA 4.0","snapshot":"2025-07-28","licence_url":"https://creativecommons.org/licenses/by-sa/4.0/legalcode","row_kind":"abstracts"},"code_links":[{"paper_slug":"distributed-memory-based-self-supervised","repo_url":"https://github.com/taewonpark/DAM","is_official":1,"mentioned_in_paper":1,"mentioned_in_github":1,"framework":"tf","reach":{"status":"ok","spdx":"MIT"}}],"tasks":[{"task_slug":"memorization","task_name":"Memorization"},{"task_slug":"question-answering","task_name":"Question Answering"},{"task_slug":"relational-reasoning","task_name":"Relational Reasoning"}],"methods":[],"datasets_introduced":[],"methods_introduced":[],"results":[],"syntology":{"atlas_url":"https://app.syntology.ai/?focus=2007.10637","mcp":{"get_harvested_code_for_paper":{"arxiv_id":"2007.10637"}},"developers":"https://syntology.ai/developers","read_at":"2026-09-24T18:15:14+00:00","read_at_is":"when the build read Syntology's graph, not when any sample ran","claim":"Per-sample execution status on synthesized fixtures; not a correctness claim about the paper. Samples come from repositories linked to the paper, official or community; repo_kind says which.","repos":[{"provenance":"external:paperswithcode_snapshot_2025-07-28","url":"https://github.com/taewonpark/DAM","reach":{"status":"ok","spdx":"MIT"}}],"summary":{"ran_draft_wrong":1,"unverified":8},"by_repo_kind":{"official":{"samples":9,"ran":1,"repositories":1}},"repo_kind_vocabulary":{"official":"The archive marks this repository official for the paper","named_in_paper":"The archive records that the paper mentions this repository; it is not marked official","listed":"In the archive's code links for this paper, not marked official and not recorded as mentioned in the paper","found_in_text":"Syntology found this repository in the paper's own text; whether it is the authors' implementation is not asserted","community":"Not in the archive's code links for this paper; a community repository Syntology harvested"},"n_pointer_only_for_licence":0,"samples":[{"code_sha256_prefix":"c03957231341007b","entry":"load","repo":"taewonpark/DAM","repo_kind":"official","path":"loader.py","file_url":"https://github.com/taewonpark/DAM/blob/HEAD/loader.py","link_basis":"plan_row","language":"python","status":"ran_draft_wrong","verification_level":1,"contract_check":"OUTPUT_MISDECLARED","metamorphic_tier":"deterministic","behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c03957231341007b"}},{"code_sha256_prefix":"327e247e1c2d7c22","entry":"bAbI_preprocessing","repo":"taewonpark/DAM","repo_kind":"official","path":"preprocess.py","file_url":"https://github.com/taewonpark/DAM/blob/HEAD/preprocess.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"327e247e1c2d7c22"}},{"code_sha256_prefix":"6b5350550ae51f2a","entry":"batch_gather","repo":"taewonpark/DAM","repo_kind":"official","path":"model/DAM/util.py","file_url":"https://github.com/taewonpark/DAM/blob/HEAD/model/DAM/util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"6b5350550ae51f2a"}},{"code_sha256_prefix":"c3d691c594f2aee2","entry":"batch_gather","repo":"taewonpark/DAM","repo_kind":"official","path":"model/DAM_test/util.py","file_url":"https://github.com/taewonpark/DAM/blob/HEAD/model/DAM_test/util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"c3d691c594f2aee2"}},{"code_sha256_prefix":"141bb8faa6e80be1","entry":"batch_invert_permutation","repo":"taewonpark/DAM","repo_kind":"official","path":"model/DAM/util.py","file_url":"https://github.com/taewonpark/DAM/blob/HEAD/model/DAM/util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"141bb8faa6e80be1"}},{"code_sha256_prefix":"0f142ba266f46ddc","entry":"create_dictionary","repo":"taewonpark/DAM","repo_kind":"official","path":"preprocess.py","file_url":"https://github.com/taewonpark/DAM/blob/HEAD/preprocess.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"0f142ba266f46ddc"}},{"code_sha256_prefix":"acbb1e971209606d","entry":"encode_data","repo":"taewonpark/DAM","repo_kind":"official","path":"preprocess.py","file_url":"https://github.com/taewonpark/DAM/blob/HEAD/preprocess.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"acbb1e971209606d"}},{"code_sha256_prefix":"eb2b8d997f4e7238","entry":"one_hot","repo":"taewonpark/DAM","repo_kind":"official","path":"model/DAM/util.py","file_url":"https://github.com/taewonpark/DAM/blob/HEAD/model/DAM/util.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"eb2b8d997f4e7238"}},{"code_sha256_prefix":"67991073de818e3c","entry":"weighted_softmax","repo":"taewonpark/DAM","repo_kind":"official","path":"model/DAM/addressing.py","file_url":"https://github.com/taewonpark/DAM/blob/HEAD/model/DAM/addressing.py","link_basis":"first_harvest_node","language":"python","status":"unverified","verification_level":0,"contract_check":null,"metamorphic_tier":null,"behaviour_fingerprint":false,"licence":"MIT","inline_ok":true,"mcp_get_code":{"code_sha256":"67991073de818e3c"}}]},"arxiv_metadata":null,"syntology_extracted_results":null}